2023-02-13 22:00:10,167 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-13 22:00:10,382 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b 2023-02-13 22:00:10,395 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 22:00:10,395 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3, deleteOnExit=true 2023-02-13 22:00:10,396 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 22:00:10,396 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/test.cache.data in system properties and HBase conf 2023-02-13 22:00:10,397 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 22:00:10,397 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/hadoop.log.dir in system properties and HBase conf 2023-02-13 22:00:10,397 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 22:00:10,398 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 22:00:10,398 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 22:00:10,496 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 22:00:10,835 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 22:00:10,840 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 22:00:10,840 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 22:00:10,840 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 22:00:10,840 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 22:00:10,841 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 22:00:10,841 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 22:00:10,841 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 22:00:10,842 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 22:00:10,842 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 22:00:10,842 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/nfs.dump.dir in system properties and HBase conf 2023-02-13 22:00:10,843 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/java.io.tmpdir in system properties and HBase conf 2023-02-13 22:00:10,843 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 22:00:10,844 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 22:00:10,844 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 22:00:11,298 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 22:00:11,302 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 22:00:11,543 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 22:00:11,708 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 22:00:11,727 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 22:00:11,766 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 22:00:11,793 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/java.io.tmpdir/Jetty_localhost_localdomain_45131_hdfs____.cj327s/webapp 2023-02-13 22:00:11,924 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:45131 2023-02-13 22:00:11,932 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 22:00:11,932 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 22:00:12,266 WARN [Listener at localhost.localdomain/37463] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 22:00:12,376 WARN [Listener at localhost.localdomain/37463] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 22:00:12,395 WARN [Listener at localhost.localdomain/37463] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 22:00:12,401 INFO [Listener at localhost.localdomain/37463] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 22:00:12,405 INFO [Listener at localhost.localdomain/37463] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/java.io.tmpdir/Jetty_localhost_34315_datanode____.ua54z/webapp 2023-02-13 22:00:12,488 INFO [Listener at localhost.localdomain/37463] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34315 2023-02-13 22:00:12,774 WARN [Listener at localhost.localdomain/42401] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 22:00:12,782 WARN [Listener at localhost.localdomain/42401] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 22:00:12,787 WARN [Listener at localhost.localdomain/42401] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 22:00:12,788 INFO [Listener at localhost.localdomain/42401] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 22:00:12,795 INFO [Listener at localhost.localdomain/42401] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/java.io.tmpdir/Jetty_localhost_33687_datanode____e605oc/webapp 2023-02-13 22:00:12,878 INFO [Listener at localhost.localdomain/42401] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33687 2023-02-13 22:00:12,886 WARN [Listener at localhost.localdomain/37029] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 22:00:12,899 WARN [Listener at localhost.localdomain/37029] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 22:00:12,903 WARN [Listener at localhost.localdomain/37029] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 22:00:12,904 INFO [Listener at localhost.localdomain/37029] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 22:00:12,908 INFO [Listener at localhost.localdomain/37029] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/java.io.tmpdir/Jetty_localhost_43143_datanode____.z9bz0o/webapp 2023-02-13 22:00:13,019 INFO [Listener at localhost.localdomain/37029] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43143 2023-02-13 22:00:13,033 WARN [Listener at localhost.localdomain/42295] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 22:00:13,226 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x24b82730fd1b4b09: Processing first storage report for DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf from datanode afe7de91-8b5d-4950-ba1d-ef3326012043 2023-02-13 22:00:13,227 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x24b82730fd1b4b09: from storage DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf node DatanodeRegistration(127.0.0.1:38805, datanodeUuid=afe7de91-8b5d-4950-ba1d-ef3326012043, infoPort=35057, infoSecurePort=0, ipcPort=42295, storageInfo=lv=-57;cid=testClusterID;nsid=121195324;c=1676325611364), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 22:00:13,228 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x128ed2f8934a1d34: Processing first storage report for DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99 from datanode 643297d9-b5a4-472e-97c3-eb890073d22e 2023-02-13 22:00:13,228 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x128ed2f8934a1d34: from storage DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99 node DatanodeRegistration(127.0.0.1:33677, datanodeUuid=643297d9-b5a4-472e-97c3-eb890073d22e, infoPort=39741, infoSecurePort=0, ipcPort=37029, storageInfo=lv=-57;cid=testClusterID;nsid=121195324;c=1676325611364), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 22:00:13,228 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x268b99077bf595bd: Processing first storage report for DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73 from datanode 3f4de7bc-de25-42a6-88c4-90335eaeb4a2 2023-02-13 22:00:13,228 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x268b99077bf595bd: from storage DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73 node DatanodeRegistration(127.0.0.1:45835, datanodeUuid=3f4de7bc-de25-42a6-88c4-90335eaeb4a2, infoPort=43409, infoSecurePort=0, ipcPort=42401, storageInfo=lv=-57;cid=testClusterID;nsid=121195324;c=1676325611364), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 22:00:13,228 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x24b82730fd1b4b09: Processing first storage report for DS-eafc84a7-0d06-43e7-95c5-34d49af5c98a from datanode afe7de91-8b5d-4950-ba1d-ef3326012043 2023-02-13 22:00:13,228 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x24b82730fd1b4b09: from storage DS-eafc84a7-0d06-43e7-95c5-34d49af5c98a node DatanodeRegistration(127.0.0.1:38805, datanodeUuid=afe7de91-8b5d-4950-ba1d-ef3326012043, infoPort=35057, infoSecurePort=0, ipcPort=42295, storageInfo=lv=-57;cid=testClusterID;nsid=121195324;c=1676325611364), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 22:00:13,229 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x128ed2f8934a1d34: Processing first storage report for DS-8a7b79d8-7b2c-486a-924c-da4742a65249 from datanode 643297d9-b5a4-472e-97c3-eb890073d22e 2023-02-13 22:00:13,229 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x128ed2f8934a1d34: from storage DS-8a7b79d8-7b2c-486a-924c-da4742a65249 node DatanodeRegistration(127.0.0.1:33677, datanodeUuid=643297d9-b5a4-472e-97c3-eb890073d22e, infoPort=39741, infoSecurePort=0, ipcPort=37029, storageInfo=lv=-57;cid=testClusterID;nsid=121195324;c=1676325611364), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 22:00:13,229 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x268b99077bf595bd: Processing first storage report for DS-daaf28a9-63cf-426b-a3b5-cc4d0e1b8f5f from datanode 3f4de7bc-de25-42a6-88c4-90335eaeb4a2 2023-02-13 22:00:13,229 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x268b99077bf595bd: from storage DS-daaf28a9-63cf-426b-a3b5-cc4d0e1b8f5f node DatanodeRegistration(127.0.0.1:45835, datanodeUuid=3f4de7bc-de25-42a6-88c4-90335eaeb4a2, infoPort=43409, infoSecurePort=0, ipcPort=42401, storageInfo=lv=-57;cid=testClusterID;nsid=121195324;c=1676325611364), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 22:00:13,573 DEBUG [Listener at localhost.localdomain/42295] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b 2023-02-13 22:00:13,631 INFO [Listener at localhost.localdomain/42295] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/zookeeper_0, clientPort=63985, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 22:00:13,643 INFO [Listener at localhost.localdomain/42295] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=63985 2023-02-13 22:00:13,653 INFO [Listener at localhost.localdomain/42295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:13,655 INFO [Listener at localhost.localdomain/42295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:14,288 INFO [Listener at localhost.localdomain/42295] util.FSUtils(479): Created version file at hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c with version=8 2023-02-13 22:00:14,289 INFO [Listener at localhost.localdomain/42295] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/hbase-staging 2023-02-13 22:00:14,559 INFO [Listener at localhost.localdomain/42295] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 22:00:14,936 INFO [Listener at localhost.localdomain/42295] client.ConnectionUtils(127): master/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 22:00:14,962 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:14,962 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:14,962 INFO [Listener at localhost.localdomain/42295] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 22:00:14,963 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:14,963 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 22:00:15,081 INFO [Listener at localhost.localdomain/42295] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 22:00:15,140 DEBUG [Listener at localhost.localdomain/42295] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 22:00:15,220 INFO [Listener at localhost.localdomain/42295] ipc.NettyRpcServer(120): Bind to /148.251.75.209:36939 2023-02-13 22:00:15,229 INFO [Listener at localhost.localdomain/42295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:15,231 INFO [Listener at localhost.localdomain/42295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:15,249 INFO [Listener at localhost.localdomain/42295] zookeeper.RecoverableZooKeeper(93): Process identifier=master:36939 connecting to ZooKeeper ensemble=127.0.0.1:63985 2023-02-13 22:00:15,282 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:369390x0, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 22:00:15,283 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:36939-0x1015ddab1e10000 connected 2023-02-13 22:00:15,311 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:15,312 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:00:15,315 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 22:00:15,322 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36939 2023-02-13 22:00:15,322 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36939 2023-02-13 22:00:15,322 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36939 2023-02-13 22:00:15,323 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36939 2023-02-13 22:00:15,323 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36939 2023-02-13 22:00:15,328 INFO [Listener at localhost.localdomain/42295] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c, hbase.cluster.distributed=false 2023-02-13 22:00:15,388 INFO [Listener at localhost.localdomain/42295] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 22:00:15,388 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:15,389 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:15,389 INFO [Listener at localhost.localdomain/42295] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 22:00:15,389 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:15,389 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 22:00:15,393 INFO [Listener at localhost.localdomain/42295] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 22:00:15,396 INFO [Listener at localhost.localdomain/42295] ipc.NettyRpcServer(120): Bind to /148.251.75.209:37979 2023-02-13 22:00:15,398 INFO [Listener at localhost.localdomain/42295] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 22:00:15,404 DEBUG [Listener at localhost.localdomain/42295] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 22:00:15,405 INFO [Listener at localhost.localdomain/42295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:15,407 INFO [Listener at localhost.localdomain/42295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:15,408 INFO [Listener at localhost.localdomain/42295] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37979 connecting to ZooKeeper ensemble=127.0.0.1:63985 2023-02-13 22:00:15,413 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:379790x0, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 22:00:15,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37979-0x1015ddab1e10001 connected 2023-02-13 22:00:15,414 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:15,416 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:00:15,417 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 22:00:15,418 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37979 2023-02-13 22:00:15,418 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37979 2023-02-13 22:00:15,419 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37979 2023-02-13 22:00:15,419 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37979 2023-02-13 22:00:15,419 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37979 2023-02-13 22:00:15,430 INFO [Listener at localhost.localdomain/42295] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 22:00:15,430 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:15,430 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:15,430 INFO [Listener at localhost.localdomain/42295] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 22:00:15,431 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:15,431 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 22:00:15,431 INFO [Listener at localhost.localdomain/42295] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 22:00:15,435 INFO [Listener at localhost.localdomain/42295] ipc.NettyRpcServer(120): Bind to /148.251.75.209:33193 2023-02-13 22:00:15,436 INFO [Listener at localhost.localdomain/42295] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 22:00:15,437 DEBUG [Listener at localhost.localdomain/42295] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 22:00:15,438 INFO [Listener at localhost.localdomain/42295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:15,440 INFO [Listener at localhost.localdomain/42295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:15,442 INFO [Listener at localhost.localdomain/42295] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33193 connecting to ZooKeeper ensemble=127.0.0.1:63985 2023-02-13 22:00:15,446 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:331930x0, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 22:00:15,447 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): regionserver:331930x0, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:15,448 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): regionserver:331930x0, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:00:15,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33193-0x1015ddab1e10002 connected 2023-02-13 22:00:15,449 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 22:00:15,455 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33193 2023-02-13 22:00:15,455 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33193 2023-02-13 22:00:15,455 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33193 2023-02-13 22:00:15,456 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33193 2023-02-13 22:00:15,457 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33193 2023-02-13 22:00:15,468 INFO [Listener at localhost.localdomain/42295] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 22:00:15,468 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:15,468 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:15,468 INFO [Listener at localhost.localdomain/42295] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 22:00:15,469 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:15,469 INFO [Listener at localhost.localdomain/42295] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 22:00:15,469 INFO [Listener at localhost.localdomain/42295] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 22:00:15,470 INFO [Listener at localhost.localdomain/42295] ipc.NettyRpcServer(120): Bind to /148.251.75.209:42047 2023-02-13 22:00:15,471 INFO [Listener at localhost.localdomain/42295] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 22:00:15,475 DEBUG [Listener at localhost.localdomain/42295] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 22:00:15,476 INFO [Listener at localhost.localdomain/42295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:15,477 INFO [Listener at localhost.localdomain/42295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:15,478 INFO [Listener at localhost.localdomain/42295] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42047 connecting to ZooKeeper ensemble=127.0.0.1:63985 2023-02-13 22:00:15,482 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:420470x0, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 22:00:15,483 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): regionserver:420470x0, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:15,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:42047-0x1015ddab1e10003 connected 2023-02-13 22:00:15,484 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:00:15,485 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 22:00:15,485 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42047 2023-02-13 22:00:15,485 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42047 2023-02-13 22:00:15,486 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42047 2023-02-13 22:00:15,486 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42047 2023-02-13 22:00:15,486 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42047 2023-02-13 22:00:15,488 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase20.apache.org,36939,1676325614416 2023-02-13 22:00:15,498 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 22:00:15,500 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase20.apache.org,36939,1676325614416 2023-02-13 22:00:15,521 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:15,521 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:15,521 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:15,521 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:15,523 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:15,524 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 22:00:15,525 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase20.apache.org,36939,1676325614416 from backup master directory 2023-02-13 22:00:15,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 22:00:15,531 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase20.apache.org,36939,1676325614416 2023-02-13 22:00:15,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 22:00:15,532 WARN [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 22:00:15,532 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase20.apache.org,36939,1676325614416 2023-02-13 22:00:15,535 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 22:00:15,536 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 22:00:15,623 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/hbase.id with ID: fba8ec1d-7499-4f15-a9ea-cb38c5ef60d4 2023-02-13 22:00:15,670 INFO [master/jenkins-hbase20:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:15,686 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:15,732 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2ad34793 to 127.0.0.1:63985 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 22:00:15,763 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3af88382, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 22:00:15,784 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 22:00:15,786 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 22:00:15,800 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 22:00:15,800 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 22:00:15,802 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 22:00:15,806 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 22:00:15,807 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 22:00:15,836 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/data/master/store-tmp 2023-02-13 22:00:15,914 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:15,915 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 22:00:15,915 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:15,915 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:15,916 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 22:00:15,916 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:15,916 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:15,916 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676325615915Disabling compacts and flushes for region at 1676325615915Disabling writes for close at 1676325615916 (+1 ms)Writing region close event to WAL at 1676325615916Closed at 1676325615916 2023-02-13 22:00:15,918 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/WALs/jenkins-hbase20.apache.org,36939,1676325614416 2023-02-13 22:00:15,937 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C36939%2C1676325614416, suffix=, logDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/WALs/jenkins-hbase20.apache.org,36939,1676325614416, archiveDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/oldWALs, maxLogs=10 2023-02-13 22:00:15,999 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK] 2023-02-13 22:00:15,999 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK] 2023-02-13 22:00:15,999 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK] 2023-02-13 22:00:16,007 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 22:00:16,080 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/WALs/jenkins-hbase20.apache.org,36939,1676325614416/jenkins-hbase20.apache.org%2C36939%2C1676325614416.1676325615946 2023-02-13 22:00:16,080 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK], DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK], DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK]] 2023-02-13 22:00:16,081 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:16,082 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:16,085 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 22:00:16,087 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 22:00:16,138 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 22:00:16,145 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 22:00:16,171 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 22:00:16,183 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:16,188 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 22:00:16,190 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 22:00:16,205 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 22:00:16,209 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:16,211 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63052147, jitterRate=-0.06044979393482208}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:16,211 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676325616109Initializing all the Stores at 1676325616111 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325616112 (+1 ms)Cleaning up temporary data from old regions at 1676325616196 (+84 ms)Cleaning up detritus from prior splits at 1676325616197 (+1 ms)Region opened successfully at 1676325616211 (+14 ms) 2023-02-13 22:00:16,212 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 22:00:16,230 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:16,230 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:16,232 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:16,234 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:16,262 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 27 msec 2023-02-13 22:00:16,262 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 22:00:16,284 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 22:00:16,290 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 22:00:16,311 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 22:00:16,314 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 22:00:16,315 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 22:00:16,320 INFO [master/jenkins-hbase20:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 22:00:16,324 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 22:00:16,326 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:16,328 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 22:00:16,328 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 22:00:16,340 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 22:00:16,348 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 22:00:16,348 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 22:00:16,348 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 22:00:16,348 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 22:00:16,349 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:16,349 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase20.apache.org,36939,1676325614416, sessionid=0x1015ddab1e10000, setting cluster-up flag (Was=false) 2023-02-13 22:00:16,362 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:16,365 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 22:00:16,367 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,36939,1676325614416 2023-02-13 22:00:16,371 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:16,374 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 22:00:16,376 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,36939,1676325614416 2023-02-13 22:00:16,378 WARN [master/jenkins-hbase20:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.hbase-snapshot/.tmp 2023-02-13 22:00:16,390 INFO [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(952): ClusterId : fba8ec1d-7499-4f15-a9ea-cb38c5ef60d4 2023-02-13 22:00:16,391 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(952): ClusterId : fba8ec1d-7499-4f15-a9ea-cb38c5ef60d4 2023-02-13 22:00:16,390 INFO [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(952): ClusterId : fba8ec1d-7499-4f15-a9ea-cb38c5ef60d4 2023-02-13 22:00:16,396 DEBUG [RS:0;jenkins-hbase20:37979] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 22:00:16,396 DEBUG [RS:1;jenkins-hbase20:33193] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 22:00:16,396 DEBUG [RS:2;jenkins-hbase20:42047] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 22:00:16,401 DEBUG [RS:0;jenkins-hbase20:37979] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 22:00:16,401 DEBUG [RS:2;jenkins-hbase20:42047] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 22:00:16,401 DEBUG [RS:1;jenkins-hbase20:33193] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 22:00:16,401 DEBUG [RS:2;jenkins-hbase20:42047] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 22:00:16,401 DEBUG [RS:0;jenkins-hbase20:37979] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 22:00:16,402 DEBUG [RS:1;jenkins-hbase20:33193] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 22:00:16,404 DEBUG [RS:1;jenkins-hbase20:33193] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 22:00:16,404 DEBUG [RS:2;jenkins-hbase20:42047] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 22:00:16,406 DEBUG [RS:2;jenkins-hbase20:42047] zookeeper.ReadOnlyZKClient(139): Connect 0x25b1011f to 127.0.0.1:63985 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 22:00:16,406 DEBUG [RS:1;jenkins-hbase20:33193] zookeeper.ReadOnlyZKClient(139): Connect 0x78dcafe0 to 127.0.0.1:63985 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 22:00:16,407 DEBUG [RS:0;jenkins-hbase20:37979] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 22:00:16,409 DEBUG [RS:0;jenkins-hbase20:37979] zookeeper.ReadOnlyZKClient(139): Connect 0x28fb1460 to 127.0.0.1:63985 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 22:00:16,424 DEBUG [RS:2;jenkins-hbase20:42047] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b76fbd9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 22:00:16,425 DEBUG [RS:1;jenkins-hbase20:33193] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@761b5e8b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 22:00:16,425 DEBUG [RS:2;jenkins-hbase20:42047] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f496a6f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 22:00:16,425 DEBUG [RS:1;jenkins-hbase20:33193] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7224c490, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 22:00:16,427 DEBUG [RS:0;jenkins-hbase20:37979] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@263cff0f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 22:00:16,427 DEBUG [RS:0;jenkins-hbase20:37979] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@22037f10, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 22:00:16,446 DEBUG [RS:1;jenkins-hbase20:33193] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase20:33193 2023-02-13 22:00:16,448 DEBUG [RS:0;jenkins-hbase20:37979] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase20:37979 2023-02-13 22:00:16,449 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase20:42047 2023-02-13 22:00:16,450 INFO [RS:1;jenkins-hbase20:33193] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 22:00:16,450 INFO [RS:2;jenkins-hbase20:42047] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 22:00:16,450 INFO [RS:0;jenkins-hbase20:37979] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 22:00:16,450 INFO [RS:2;jenkins-hbase20:42047] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 22:00:16,450 INFO [RS:1;jenkins-hbase20:33193] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 22:00:16,451 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 22:00:16,450 INFO [RS:0;jenkins-hbase20:37979] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 22:00:16,451 DEBUG [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 22:00:16,451 DEBUG [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 22:00:16,453 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,36939,1676325614416 with isa=jenkins-hbase20.apache.org/148.251.75.209:42047, startcode=1676325615467 2023-02-13 22:00:16,453 INFO [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,36939,1676325614416 with isa=jenkins-hbase20.apache.org/148.251.75.209:33193, startcode=1676325615429 2023-02-13 22:00:16,453 INFO [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,36939,1676325614416 with isa=jenkins-hbase20.apache.org/148.251.75.209:37979, startcode=1676325615388 2023-02-13 22:00:16,472 DEBUG [RS:2;jenkins-hbase20:42047] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 22:00:16,472 DEBUG [RS:1;jenkins-hbase20:33193] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 22:00:16,472 DEBUG [RS:0;jenkins-hbase20:37979] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 22:00:16,521 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:16,530 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 22:00:16,530 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 22:00:16,530 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 22:00:16,530 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 22:00:16,530 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase20:0, corePoolSize=10, maxPoolSize=10 2023-02-13 22:00:16,530 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,531 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 22:00:16,531 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,532 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325646531 2023-02-13 22:00:16,533 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:53121, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 22:00:16,533 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:39679, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 22:00:16,534 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 22:00:16,533 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:34793, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 22:00:16,545 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:16,545 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 22:00:16,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36939] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 22:00:16,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 22:00:16,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36939] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 22:00:16,551 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 22:00:16,553 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 22:00:16,562 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 22:00:16,562 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 22:00:16,563 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 22:00:16,563 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 22:00:16,564 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,566 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 22:00:16,568 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 22:00:16,568 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 22:00:16,570 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 22:00:16,571 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 22:00:16,572 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325616572,5,FailOnTimeoutGroup] 2023-02-13 22:00:16,572 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325616572,5,FailOnTimeoutGroup] 2023-02-13 22:00:16,572 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,572 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 22:00:16,573 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,574 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,583 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 22:00:16,583 DEBUG [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 22:00:16,583 DEBUG [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 22:00:16,584 WARN [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 22:00:16,584 WARN [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 22:00:16,584 WARN [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 22:00:16,607 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:16,609 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:16,609 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c 2023-02-13 22:00:16,634 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:16,636 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 22:00:16,639 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/info 2023-02-13 22:00:16,640 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 22:00:16,641 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:16,641 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 22:00:16,644 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/rep_barrier 2023-02-13 22:00:16,645 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 22:00:16,646 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:16,646 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 22:00:16,648 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/table 2023-02-13 22:00:16,649 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 22:00:16,650 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:16,651 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740 2023-02-13 22:00:16,653 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740 2023-02-13 22:00:16,656 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 22:00:16,658 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 22:00:16,662 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:16,662 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70993603, jitterRate=0.05788712203502655}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 22:00:16,663 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676325616634Initializing all the Stores at 1676325616635 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325616635Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325616635Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325616635Cleaning up temporary data from old regions at 1676325616654 (+19 ms)Cleaning up detritus from prior splits at 1676325616655 (+1 ms)Region opened successfully at 1676325616663 (+8 ms) 2023-02-13 22:00:16,663 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 22:00:16,663 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 22:00:16,663 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 22:00:16,663 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 22:00:16,663 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 22:00:16,664 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 22:00:16,664 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676325616663Disabling compacts and flushes for region at 1676325616663Disabling writes for close at 1676325616663Writing region close event to WAL at 1676325616664 (+1 ms)Closed at 1676325616664 2023-02-13 22:00:16,670 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:16,670 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 22:00:16,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 22:00:16,685 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,36939,1676325614416 with isa=jenkins-hbase20.apache.org/148.251.75.209:42047, startcode=1676325615467 2023-02-13 22:00:16,685 INFO [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,36939,1676325614416 with isa=jenkins-hbase20.apache.org/148.251.75.209:33193, startcode=1676325615429 2023-02-13 22:00:16,685 INFO [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,36939,1676325614416 with isa=jenkins-hbase20.apache.org/148.251.75.209:37979, startcode=1676325615388 2023-02-13 22:00:16,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36939] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:16,693 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 22:00:16,696 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 22:00:16,696 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36939] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:16,697 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:16,699 DEBUG [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c 2023-02-13 22:00:16,699 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c 2023-02-13 22:00:16,699 DEBUG [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c 2023-02-13 22:00:16,699 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:37463 2023-02-13 22:00:16,699 DEBUG [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:37463 2023-02-13 22:00:16,700 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 22:00:16,700 DEBUG [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:37463 2023-02-13 22:00:16,700 DEBUG [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 22:00:16,700 DEBUG [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 22:00:16,705 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:16,706 DEBUG [RS:2;jenkins-hbase20:42047] zookeeper.ZKUtil(162): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:16,706 WARN [RS:2;jenkins-hbase20:42047] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 22:00:16,706 DEBUG [RS:0;jenkins-hbase20:37979] zookeeper.ZKUtil(162): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:16,706 INFO [RS:2;jenkins-hbase20:42047] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 22:00:16,706 WARN [RS:0;jenkins-hbase20:37979] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 22:00:16,707 INFO [RS:0;jenkins-hbase20:37979] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 22:00:16,707 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:16,707 DEBUG [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:16,707 DEBUG [RS:1;jenkins-hbase20:33193] zookeeper.ZKUtil(162): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:16,707 WARN [RS:1;jenkins-hbase20:33193] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 22:00:16,707 INFO [RS:1;jenkins-hbase20:33193] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 22:00:16,707 DEBUG [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:16,713 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,33193,1676325615429] 2023-02-13 22:00:16,713 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,37979,1676325615388] 2023-02-13 22:00:16,713 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,42047,1676325615467] 2023-02-13 22:00:16,726 DEBUG [RS:1;jenkins-hbase20:33193] zookeeper.ZKUtil(162): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:16,726 DEBUG [RS:0;jenkins-hbase20:37979] zookeeper.ZKUtil(162): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:16,726 DEBUG [RS:2;jenkins-hbase20:42047] zookeeper.ZKUtil(162): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:16,727 DEBUG [RS:1;jenkins-hbase20:33193] zookeeper.ZKUtil(162): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:16,727 DEBUG [RS:0;jenkins-hbase20:37979] zookeeper.ZKUtil(162): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:16,727 DEBUG [RS:2;jenkins-hbase20:42047] zookeeper.ZKUtil(162): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:16,727 DEBUG [RS:1;jenkins-hbase20:33193] zookeeper.ZKUtil(162): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:16,727 DEBUG [RS:0;jenkins-hbase20:37979] zookeeper.ZKUtil(162): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:16,727 DEBUG [RS:2;jenkins-hbase20:42047] zookeeper.ZKUtil(162): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:16,735 DEBUG [RS:1;jenkins-hbase20:33193] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 22:00:16,735 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 22:00:16,735 DEBUG [RS:0;jenkins-hbase20:37979] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 22:00:16,744 INFO [RS:2;jenkins-hbase20:42047] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 22:00:16,744 INFO [RS:0;jenkins-hbase20:37979] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 22:00:16,744 INFO [RS:1;jenkins-hbase20:33193] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 22:00:16,766 INFO [RS:0;jenkins-hbase20:37979] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 22:00:16,766 INFO [RS:2;jenkins-hbase20:42047] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 22:00:16,766 INFO [RS:1;jenkins-hbase20:33193] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 22:00:16,769 INFO [RS:0;jenkins-hbase20:37979] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 22:00:16,769 INFO [RS:2;jenkins-hbase20:42047] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 22:00:16,770 INFO [RS:0;jenkins-hbase20:37979] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,769 INFO [RS:1;jenkins-hbase20:33193] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 22:00:16,770 INFO [RS:2;jenkins-hbase20:42047] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,771 INFO [RS:1;jenkins-hbase20:33193] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,772 INFO [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 22:00:16,772 INFO [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 22:00:16,772 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 22:00:16,780 INFO [RS:2;jenkins-hbase20:42047] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,780 INFO [RS:0;jenkins-hbase20:37979] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,780 INFO [RS:1;jenkins-hbase20:33193] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,780 DEBUG [RS:2;jenkins-hbase20:42047] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,780 DEBUG [RS:1;jenkins-hbase20:33193] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,780 DEBUG [RS:0;jenkins-hbase20:37979] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,780 DEBUG [RS:1;jenkins-hbase20:33193] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,780 DEBUG [RS:0;jenkins-hbase20:37979] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:1;jenkins-hbase20:33193] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:0;jenkins-hbase20:37979] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,780 DEBUG [RS:2;jenkins-hbase20:42047] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:0;jenkins-hbase20:37979] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:1;jenkins-hbase20:33193] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:0;jenkins-hbase20:37979] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:1;jenkins-hbase20:33193] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:0;jenkins-hbase20:37979] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 22:00:16,781 DEBUG [RS:1;jenkins-hbase20:33193] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 22:00:16,781 DEBUG [RS:2;jenkins-hbase20:42047] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:1;jenkins-hbase20:33193] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:2;jenkins-hbase20:42047] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:0;jenkins-hbase20:37979] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:2;jenkins-hbase20:42047] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:1;jenkins-hbase20:33193] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:2;jenkins-hbase20:42047] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 22:00:16,782 DEBUG [RS:1;jenkins-hbase20:33193] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,782 DEBUG [RS:2;jenkins-hbase20:42047] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,781 DEBUG [RS:0;jenkins-hbase20:37979] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,782 DEBUG [RS:2;jenkins-hbase20:42047] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,782 DEBUG [RS:1;jenkins-hbase20:33193] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,782 DEBUG [RS:2;jenkins-hbase20:42047] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,782 DEBUG [RS:0;jenkins-hbase20:37979] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,782 DEBUG [RS:2;jenkins-hbase20:42047] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,782 DEBUG [RS:0;jenkins-hbase20:37979] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:16,783 INFO [RS:1;jenkins-hbase20:33193] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,783 INFO [RS:1;jenkins-hbase20:33193] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,783 INFO [RS:1;jenkins-hbase20:33193] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,785 INFO [RS:0;jenkins-hbase20:37979] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,785 INFO [RS:2;jenkins-hbase20:42047] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,786 INFO [RS:0;jenkins-hbase20:37979] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,786 INFO [RS:2;jenkins-hbase20:42047] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,786 INFO [RS:0;jenkins-hbase20:37979] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,786 INFO [RS:2;jenkins-hbase20:42047] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,799 INFO [RS:2;jenkins-hbase20:42047] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 22:00:16,799 INFO [RS:1;jenkins-hbase20:33193] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 22:00:16,799 INFO [RS:0;jenkins-hbase20:37979] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 22:00:16,802 INFO [RS:2;jenkins-hbase20:42047] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,42047,1676325615467-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,802 INFO [RS:0;jenkins-hbase20:37979] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,37979,1676325615388-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,802 INFO [RS:1;jenkins-hbase20:33193] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,33193,1676325615429-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:16,817 INFO [RS:1;jenkins-hbase20:33193] regionserver.Replication(203): jenkins-hbase20.apache.org,33193,1676325615429 started 2023-02-13 22:00:16,817 INFO [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,33193,1676325615429, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:33193, sessionid=0x1015ddab1e10002 2023-02-13 22:00:16,818 INFO [RS:0;jenkins-hbase20:37979] regionserver.Replication(203): jenkins-hbase20.apache.org,37979,1676325615388 started 2023-02-13 22:00:16,818 DEBUG [RS:1;jenkins-hbase20:33193] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 22:00:16,818 INFO [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,37979,1676325615388, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:37979, sessionid=0x1015ddab1e10001 2023-02-13 22:00:16,818 DEBUG [RS:1;jenkins-hbase20:33193] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:16,818 DEBUG [RS:0;jenkins-hbase20:37979] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 22:00:16,818 DEBUG [RS:1;jenkins-hbase20:33193] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,33193,1676325615429' 2023-02-13 22:00:16,818 DEBUG [RS:0;jenkins-hbase20:37979] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:16,819 DEBUG [RS:1;jenkins-hbase20:33193] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:16,819 DEBUG [RS:0;jenkins-hbase20:37979] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,37979,1676325615388' 2023-02-13 22:00:16,819 DEBUG [RS:0;jenkins-hbase20:37979] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:16,820 DEBUG [RS:0;jenkins-hbase20:37979] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:16,820 DEBUG [RS:1;jenkins-hbase20:33193] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:16,820 DEBUG [RS:0;jenkins-hbase20:37979] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 22:00:16,820 DEBUG [RS:1;jenkins-hbase20:33193] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 22:00:16,820 DEBUG [RS:0;jenkins-hbase20:37979] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 22:00:16,820 DEBUG [RS:1;jenkins-hbase20:33193] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 22:00:16,820 DEBUG [RS:0;jenkins-hbase20:37979] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:16,820 DEBUG [RS:1;jenkins-hbase20:33193] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:16,821 DEBUG [RS:0;jenkins-hbase20:37979] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,37979,1676325615388' 2023-02-13 22:00:16,821 DEBUG [RS:1;jenkins-hbase20:33193] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,33193,1676325615429' 2023-02-13 22:00:16,821 DEBUG [RS:1;jenkins-hbase20:33193] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 22:00:16,821 DEBUG [RS:0;jenkins-hbase20:37979] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 22:00:16,821 INFO [RS:2;jenkins-hbase20:42047] regionserver.Replication(203): jenkins-hbase20.apache.org,42047,1676325615467 started 2023-02-13 22:00:16,821 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,42047,1676325615467, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:42047, sessionid=0x1015ddab1e10003 2023-02-13 22:00:16,821 DEBUG [RS:2;jenkins-hbase20:42047] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 22:00:16,821 DEBUG [RS:0;jenkins-hbase20:37979] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 22:00:16,821 DEBUG [RS:2;jenkins-hbase20:42047] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:16,822 DEBUG [RS:2;jenkins-hbase20:42047] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,42047,1676325615467' 2023-02-13 22:00:16,822 DEBUG [RS:2;jenkins-hbase20:42047] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:16,822 DEBUG [RS:0;jenkins-hbase20:37979] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 22:00:16,822 INFO [RS:0;jenkins-hbase20:37979] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 22:00:16,822 INFO [RS:0;jenkins-hbase20:37979] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 22:00:16,822 DEBUG [RS:2;jenkins-hbase20:42047] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:16,822 DEBUG [RS:2;jenkins-hbase20:42047] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 22:00:16,822 DEBUG [RS:2;jenkins-hbase20:42047] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 22:00:16,822 DEBUG [RS:2;jenkins-hbase20:42047] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:16,823 DEBUG [RS:2;jenkins-hbase20:42047] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,42047,1676325615467' 2023-02-13 22:00:16,823 DEBUG [RS:2;jenkins-hbase20:42047] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 22:00:16,823 DEBUG [RS:1;jenkins-hbase20:33193] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 22:00:16,823 DEBUG [RS:2;jenkins-hbase20:42047] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 22:00:16,823 DEBUG [RS:1;jenkins-hbase20:33193] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 22:00:16,823 INFO [RS:1;jenkins-hbase20:33193] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 22:00:16,824 INFO [RS:1;jenkins-hbase20:33193] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 22:00:16,825 DEBUG [RS:2;jenkins-hbase20:42047] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 22:00:16,825 INFO [RS:2;jenkins-hbase20:42047] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 22:00:16,825 INFO [RS:2;jenkins-hbase20:42047] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 22:00:16,848 DEBUG [jenkins-hbase20:36939] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 22:00:16,851 DEBUG [jenkins-hbase20:36939] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:16,856 DEBUG [jenkins-hbase20:36939] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:16,856 DEBUG [jenkins-hbase20:36939] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:16,856 DEBUG [jenkins-hbase20:36939] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:16,856 DEBUG [jenkins-hbase20:36939] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:16,858 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,42047,1676325615467, state=OPENING 2023-02-13 22:00:16,864 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 22:00:16,865 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:16,866 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 22:00:16,869 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:00:16,934 INFO [RS:0;jenkins-hbase20:37979] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C37979%2C1676325615388, suffix=, logDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,37979,1676325615388, archiveDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/oldWALs, maxLogs=32 2023-02-13 22:00:16,934 INFO [RS:2;jenkins-hbase20:42047] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C42047%2C1676325615467, suffix=, logDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,42047,1676325615467, archiveDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/oldWALs, maxLogs=32 2023-02-13 22:00:16,934 INFO [RS:1;jenkins-hbase20:33193] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C33193%2C1676325615429, suffix=, logDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,33193,1676325615429, archiveDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/oldWALs, maxLogs=32 2023-02-13 22:00:16,955 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK] 2023-02-13 22:00:16,956 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK] 2023-02-13 22:00:16,956 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK] 2023-02-13 22:00:16,957 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK] 2023-02-13 22:00:16,957 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK] 2023-02-13 22:00:16,957 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK] 2023-02-13 22:00:16,967 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK] 2023-02-13 22:00:16,967 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK] 2023-02-13 22:00:16,967 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK] 2023-02-13 22:00:16,968 INFO [RS:0;jenkins-hbase20:37979] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,37979,1676325615388/jenkins-hbase20.apache.org%2C37979%2C1676325615388.1676325616937 2023-02-13 22:00:16,976 DEBUG [RS:0;jenkins-hbase20:37979] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK], DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK], DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK]] 2023-02-13 22:00:16,977 INFO [RS:2;jenkins-hbase20:42047] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,42047,1676325615467/jenkins-hbase20.apache.org%2C42047%2C1676325615467.1676325616937 2023-02-13 22:00:16,977 INFO [RS:1;jenkins-hbase20:33193] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,33193,1676325615429/jenkins-hbase20.apache.org%2C33193%2C1676325615429.1676325616937 2023-02-13 22:00:16,978 DEBUG [RS:2;jenkins-hbase20:42047] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK], DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK], DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK]] 2023-02-13 22:00:16,978 DEBUG [RS:1;jenkins-hbase20:33193] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK], DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK], DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK]] 2023-02-13 22:00:17,049 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:17,051 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 22:00:17,053 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:45768, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 22:00:17,066 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 22:00:17,067 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 22:00:17,071 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C42047%2C1676325615467.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,42047,1676325615467, archiveDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/oldWALs, maxLogs=32 2023-02-13 22:00:17,090 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK] 2023-02-13 22:00:17,092 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK] 2023-02-13 22:00:17,093 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK] 2023-02-13 22:00:17,100 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/WALs/jenkins-hbase20.apache.org,42047,1676325615467/jenkins-hbase20.apache.org%2C42047%2C1676325615467.meta.1676325617072.meta 2023-02-13 22:00:17,101 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK], DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK], DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK]] 2023-02-13 22:00:17,101 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:17,105 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:17,106 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 22:00:17,119 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 22:00:17,122 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 22:00:17,125 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 22:00:17,126 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:17,126 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 22:00:17,126 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 22:00:17,128 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 22:00:17,130 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/info 2023-02-13 22:00:17,130 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/info 2023-02-13 22:00:17,131 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 22:00:17,132 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:17,132 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 22:00:17,133 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/rep_barrier 2023-02-13 22:00:17,133 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/rep_barrier 2023-02-13 22:00:17,134 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 22:00:17,135 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:17,135 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 22:00:17,137 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/table 2023-02-13 22:00:17,137 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/table 2023-02-13 22:00:17,138 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 22:00:17,139 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:17,141 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740 2023-02-13 22:00:17,144 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740 2023-02-13 22:00:17,147 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 22:00:17,149 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 22:00:17,151 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74126777, jitterRate=0.10457505285739899}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 22:00:17,151 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676325617126Writing region info on filesystem at 1676325617126Initializing all the Stores at 1676325617128 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325617128Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325617128Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325617128Cleaning up temporary data from old regions at 1676325617145 (+17 ms)Cleaning up detritus from prior splits at 1676325617146 (+1 ms)Running coprocessor post-open hooks at 1676325617151 (+5 ms)Region opened successfully at 1676325617151 2023-02-13 22:00:17,159 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676325617042 2023-02-13 22:00:17,176 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 22:00:17,177 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 22:00:17,178 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,42047,1676325615467, state=OPEN 2023-02-13 22:00:17,181 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 22:00:17,181 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 22:00:17,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 22:00:17,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,42047,1676325615467 in 312 msec 2023-02-13 22:00:17,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 22:00:17,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 510 msec 2023-02-13 22:00:17,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 761 msec 2023-02-13 22:00:17,200 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676325615518, completionTime=-1 2023-02-13 22:00:17,200 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 22:00:17,200 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:17,260 DEBUG [hconnection-0x1357b8ff-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 22:00:17,263 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:45780, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 22:00:17,279 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:17,280 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325677280 2023-02-13 22:00:17,280 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325737280 2023-02-13 22:00:17,280 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 79 msec 2023-02-13 22:00:17,302 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,36939,1676325614416-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:17,302 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,36939,1676325614416-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:17,302 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,36939,1676325614416-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:17,305 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase20:36939, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:17,307 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:17,316 DEBUG [master/jenkins-hbase20:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 22:00:17,323 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 22:00:17,324 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 22:00:17,334 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:17,336 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:00:17,339 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:00:17,361 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a 2023-02-13 22:00:17,363 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a empty. 2023-02-13 22:00:17,364 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a 2023-02-13 22:00:17,364 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 22:00:17,402 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:17,404 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 09ae03d8301e349bcc2ee47a4f5f6b0a, NAME => 'hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:00:17,426 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:17,426 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 09ae03d8301e349bcc2ee47a4f5f6b0a, disabling compactions & flushes 2023-02-13 22:00:17,426 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:00:17,426 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:00:17,427 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. after waiting 0 ms 2023-02-13 22:00:17,427 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:00:17,427 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:00:17,427 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 09ae03d8301e349bcc2ee47a4f5f6b0a: Waiting for close lock at 1676325617426Disabling compacts and flushes for region at 1676325617426Disabling writes for close at 1676325617427 (+1 ms)Writing region close event to WAL at 1676325617427Closed at 1676325617427 2023-02-13 22:00:17,433 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:00:17,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325617436"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325617436"}]},"ts":"1676325617436"} 2023-02-13 22:00:17,473 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:00:17,475 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:00:17,479 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325617475"}]},"ts":"1676325617475"} 2023-02-13 22:00:17,483 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 22:00:17,486 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:17,487 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:17,487 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:17,487 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:17,487 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:17,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=09ae03d8301e349bcc2ee47a4f5f6b0a, ASSIGN}] 2023-02-13 22:00:17,493 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=09ae03d8301e349bcc2ee47a4f5f6b0a, ASSIGN 2023-02-13 22:00:17,494 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=09ae03d8301e349bcc2ee47a4f5f6b0a, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,42047,1676325615467; forceNewPlan=false, retain=false 2023-02-13 22:00:17,646 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:17,647 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:17,647 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325617647"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325617647"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325617647"}]},"ts":"1676325617647"} 2023-02-13 22:00:17,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 09ae03d8301e349bcc2ee47a4f5f6b0a, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:00:17,815 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:00:17,818 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 09ae03d8301e349bcc2ee47a4f5f6b0a, NAME => 'hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:17,819 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:17,820 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 09ae03d8301e349bcc2ee47a4f5f6b0a 2023-02-13 22:00:17,820 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:17,820 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 09ae03d8301e349bcc2ee47a4f5f6b0a 2023-02-13 22:00:17,820 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 09ae03d8301e349bcc2ee47a4f5f6b0a 2023-02-13 22:00:17,823 INFO [StoreOpener-09ae03d8301e349bcc2ee47a4f5f6b0a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 09ae03d8301e349bcc2ee47a4f5f6b0a 2023-02-13 22:00:17,826 DEBUG [StoreOpener-09ae03d8301e349bcc2ee47a4f5f6b0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a/info 2023-02-13 22:00:17,826 DEBUG [StoreOpener-09ae03d8301e349bcc2ee47a4f5f6b0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a/info 2023-02-13 22:00:17,827 INFO [StoreOpener-09ae03d8301e349bcc2ee47a4f5f6b0a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09ae03d8301e349bcc2ee47a4f5f6b0a columnFamilyName info 2023-02-13 22:00:17,828 INFO [StoreOpener-09ae03d8301e349bcc2ee47a4f5f6b0a-1] regionserver.HStore(310): Store=09ae03d8301e349bcc2ee47a4f5f6b0a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:17,830 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a 2023-02-13 22:00:17,831 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a 2023-02-13 22:00:17,836 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 09ae03d8301e349bcc2ee47a4f5f6b0a 2023-02-13 22:00:17,839 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:17,840 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 09ae03d8301e349bcc2ee47a4f5f6b0a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74636745, jitterRate=0.11217416822910309}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:17,840 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 09ae03d8301e349bcc2ee47a4f5f6b0a: Running coprocessor pre-open hook at 1676325617820Writing region info on filesystem at 1676325617820Initializing all the Stores at 1676325617822 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325617822Cleaning up temporary data from old regions at 1676325617832 (+10 ms)Cleaning up detritus from prior splits at 1676325617833 (+1 ms)Running coprocessor post-open hooks at 1676325617840 (+7 ms)Region opened successfully at 1676325617840 2023-02-13 22:00:17,843 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a., pid=6, masterSystemTime=1676325617806 2023-02-13 22:00:17,848 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:00:17,848 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:00:17,850 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:17,851 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325617849"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325617849"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325617849"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325617849"}]},"ts":"1676325617849"} 2023-02-13 22:00:17,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 22:00:17,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 09ae03d8301e349bcc2ee47a4f5f6b0a, server=jenkins-hbase20.apache.org,42047,1676325615467 in 204 msec 2023-02-13 22:00:17,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 22:00:17,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=09ae03d8301e349bcc2ee47a4f5f6b0a, ASSIGN in 371 msec 2023-02-13 22:00:17,867 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:00:17,867 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325617867"}]},"ts":"1676325617867"} 2023-02-13 22:00:17,870 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 22:00:17,873 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:00:17,877 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 548 msec 2023-02-13 22:00:17,937 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 22:00:17,938 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 22:00:17,938 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:17,972 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:17,991 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 22:00:17,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 31 msec 2023-02-13 22:00:18,006 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:18,020 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 22:00:18,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-02-13 22:00:18,040 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 22:00:18,041 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 22:00:18,042 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.509sec 2023-02-13 22:00:18,044 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 22:00:18,046 INFO [master/jenkins-hbase20:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 22:00:18,046 INFO [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 22:00:18,048 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,36939,1676325614416-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 22:00:18,050 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,36939,1676325614416-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 22:00:18,058 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 22:00:18,096 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ReadOnlyZKClient(139): Connect 0x1753ea84 to 127.0.0.1:63985 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 22:00:18,101 DEBUG [Listener at localhost.localdomain/42295] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4eee752b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 22:00:18,114 DEBUG [hconnection-0x4e26826a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 22:00:18,125 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:45782, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 22:00:18,137 INFO [Listener at localhost.localdomain/42295] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase20.apache.org,36939,1676325614416 2023-02-13 22:00:18,166 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=356, OpenFileDescriptor=691, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=164, AvailableMemoryMB=7829 2023-02-13 22:00:18,169 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:18,177 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 22:00:18,180 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:54362, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 22:00:18,193 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:00:18,193 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:18,194 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-13 22:00:18,200 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,202 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:00:18,204 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:00:18,207 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:18,208 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835 empty. 2023-02-13 22:00:18,210 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:18,210 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 22:00:18,235 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:18,239 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 091ee0643bbb09cdba3c6e8d5b8e5835, NAME => 'testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:00:18,260 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:18,260 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 091ee0643bbb09cdba3c6e8d5b8e5835, disabling compactions & flushes 2023-02-13 22:00:18,260 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:18,260 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:18,260 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. after waiting 0 ms 2023-02-13 22:00:18,260 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:18,260 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:18,260 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 091ee0643bbb09cdba3c6e8d5b8e5835: Waiting for close lock at 1676325618260Disabling compacts and flushes for region at 1676325618260Disabling writes for close at 1676325618260Writing region close event to WAL at 1676325618260Closed at 1676325618260 2023-02-13 22:00:18,264 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:00:18,266 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676325618266"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325618266"}]},"ts":"1676325618266"} 2023-02-13 22:00:18,269 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:00:18,270 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:00:18,270 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325618270"}]},"ts":"1676325618270"} 2023-02-13 22:00:18,273 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-13 22:00:18,276 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:18,278 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:18,278 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:18,278 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:18,278 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:18,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=091ee0643bbb09cdba3c6e8d5b8e5835, ASSIGN}] 2023-02-13 22:00:18,282 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=091ee0643bbb09cdba3c6e8d5b8e5835, ASSIGN 2023-02-13 22:00:18,283 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=091ee0643bbb09cdba3c6e8d5b8e5835, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,42047,1676325615467; forceNewPlan=false, retain=false 2023-02-13 22:00:18,433 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:18,434 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=091ee0643bbb09cdba3c6e8d5b8e5835, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,435 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676325618434"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325618434"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325618434"}]},"ts":"1676325618434"} 2023-02-13 22:00:18,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 091ee0643bbb09cdba3c6e8d5b8e5835, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:00:18,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:18,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 091ee0643bbb09cdba3c6e8d5b8e5835, NAME => 'testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:18,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:18,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:18,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:18,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:18,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:18,605 INFO [StoreOpener-091ee0643bbb09cdba3c6e8d5b8e5835-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:18,607 DEBUG [StoreOpener-091ee0643bbb09cdba3c6e8d5b8e5835-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1 2023-02-13 22:00:18,608 DEBUG [StoreOpener-091ee0643bbb09cdba3c6e8d5b8e5835-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1 2023-02-13 22:00:18,608 INFO [StoreOpener-091ee0643bbb09cdba3c6e8d5b8e5835-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 091ee0643bbb09cdba3c6e8d5b8e5835 columnFamilyName cf1 2023-02-13 22:00:18,609 INFO [StoreOpener-091ee0643bbb09cdba3c6e8d5b8e5835-1] regionserver.HStore(310): Store=091ee0643bbb09cdba3c6e8d5b8e5835/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:18,609 INFO [StoreOpener-091ee0643bbb09cdba3c6e8d5b8e5835-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:18,611 DEBUG [StoreOpener-091ee0643bbb09cdba3c6e8d5b8e5835-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2 2023-02-13 22:00:18,611 DEBUG [StoreOpener-091ee0643bbb09cdba3c6e8d5b8e5835-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2 2023-02-13 22:00:18,612 INFO [StoreOpener-091ee0643bbb09cdba3c6e8d5b8e5835-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 091ee0643bbb09cdba3c6e8d5b8e5835 columnFamilyName cf2 2023-02-13 22:00:18,613 INFO [StoreOpener-091ee0643bbb09cdba3c6e8d5b8e5835-1] regionserver.HStore(310): Store=091ee0643bbb09cdba3c6e8d5b8e5835/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:18,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:18,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:18,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:18,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:18,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:18,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 091ee0643bbb09cdba3c6e8d5b8e5835; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68082872, jitterRate=0.014513850212097168}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:18,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 091ee0643bbb09cdba3c6e8d5b8e5835: Running coprocessor pre-open hook at 1676325618601Writing region info on filesystem at 1676325618601Initializing all the Stores at 1676325618602 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325618602Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325618605 (+3 ms)Cleaning up temporary data from old regions at 1676325618618 (+13 ms)Cleaning up detritus from prior splits at 1676325618619 (+1 ms)Running coprocessor post-open hooks at 1676325618627 (+8 ms)Region opened successfully at 1676325618627 2023-02-13 22:00:18,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835., pid=11, masterSystemTime=1676325618593 2023-02-13 22:00:18,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:18,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:18,632 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=091ee0643bbb09cdba3c6e8d5b8e5835, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,633 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676325618632"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325618632"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325618632"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325618632"}]},"ts":"1676325618632"} 2023-02-13 22:00:18,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 22:00:18,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 091ee0643bbb09cdba3c6e8d5b8e5835, server=jenkins-hbase20.apache.org,42047,1676325615467 in 198 msec 2023-02-13 22:00:18,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 22:00:18,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=091ee0643bbb09cdba3c6e8d5b8e5835, ASSIGN in 361 msec 2023-02-13 22:00:18,646 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:00:18,647 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325618646"}]},"ts":"1676325618646"} 2023-02-13 22:00:18,649 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-13 22:00:18,653 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:00:18,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 455 msec 2023-02-13 22:00:18,730 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:18,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,767 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:18,767 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:18,767 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 22:00:18,767 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 22:00:18,771 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,772 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,772 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:18,773 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:18,773 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:18,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:18,773 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:18,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:18,773 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,773 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,774 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,775 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,776 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:18,776 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:18,776 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:18,776 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:18,777 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:18,776 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:18,777 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:18,777 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:18,778 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:18,778 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:18,778 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:18,778 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:18,778 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:18,778 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:18,778 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:18,778 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:18,778 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. started... 2023-02-13 22:00:18,780 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 091ee0643bbb09cdba3c6e8d5b8e5835 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 22:00:18,781 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,781 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,782 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,782 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:18,782 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,782 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:18,844 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/9bca9e48275e41d39aa4b5b565d7e558 2023-02-13 22:00:18,927 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/a3798dcfb8f547bbafdd4bf70501a8fc 2023-02-13 22:00:18,941 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/9bca9e48275e41d39aa4b5b565d7e558 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558 2023-02-13 22:00:18,952 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:00:18,955 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/a3798dcfb8f547bbafdd4bf70501a8fc as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc 2023-02-13 22:00:18,969 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:00:18,971 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 091ee0643bbb09cdba3c6e8d5b8e5835 in 191ms, sequenceid=6, compaction requested=false 2023-02-13 22:00:18,973 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:18,975 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 091ee0643bbb09cdba3c6e8d5b8e5835: 2023-02-13 22:00:18,975 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:18,976 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:18,977 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:18,977 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,977 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:18,978 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:18,979 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,979 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:18,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:18,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:18,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:18,982 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,982 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:18,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:18,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:18,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:18,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 22:00:18,984 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:18,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f651fa8[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:18,984 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,988 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,989 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:18,989 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,989 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:18,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,989 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:18,989 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:18,989 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:18,989 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:18,989 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:18,989 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:18,989 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:18,989 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:18,991 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:18,992 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:18,992 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:18,992 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,992 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:18,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:18,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:18,992 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:18,992 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:18,992 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:18,992 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:18,993 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:18,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:18,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:18,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:18,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:18,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:18,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:18,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:18,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:18,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:00:18,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:18,998 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 22:00:18,998 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:18,998 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:18,998 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:18,998 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:19,005 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,005 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,005 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,005 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,005 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,005 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:19,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,006 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,005 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:19,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,006 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,007 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:19,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:19,009 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,009 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,010 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,023 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,023 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,023 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,023 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,023 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,023 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,023 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,023 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,024 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,024 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:19,023 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,025 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:19,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,024 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,025 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,025 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,025 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,025 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,025 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,025 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,025 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,027 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:19,027 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:19,127 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:19,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:19,160 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:19,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:19,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,163 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:19,163 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,164 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 22:00:19,164 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 22:00:19,165 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,165 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,166 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,166 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,166 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,166 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,167 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,167 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:19,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:19,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,169 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:19,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,169 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:19,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,170 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,170 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:19,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,170 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:19,170 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:19,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,170 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,170 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:19,170 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:19,171 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:19,171 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,171 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,171 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:19,172 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,171 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:19,172 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:19,172 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:19,172 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:19,172 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:19,172 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:19,172 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:19,172 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:19,172 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:19,173 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. started... 2023-02-13 22:00:19,173 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 091ee0643bbb09cdba3c6e8d5b8e5835 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:19,180 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,180 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,180 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:19,184 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,185 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,185 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:19,209 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/1b8a523c71b74209b082af1a0870ccde 2023-02-13 22:00:19,245 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/ce34be6674484090a2762ff472da269e 2023-02-13 22:00:19,259 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/1b8a523c71b74209b082af1a0870ccde as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde 2023-02-13 22:00:19,272 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:00:19,276 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/ce34be6674484090a2762ff472da269e as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e 2023-02-13 22:00:19,285 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:00:19,286 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 091ee0643bbb09cdba3c6e8d5b8e5835 in 113ms, sequenceid=19, compaction requested=false 2023-02-13 22:00:19,286 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 091ee0643bbb09cdba3c6e8d5b8e5835: 2023-02-13 22:00:19,286 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:19,286 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:19,287 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:19,287 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,287 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:19,287 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:19,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,289 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:19,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:19,290 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,290 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:19,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:19,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:19,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:19,293 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 22:00:19,293 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@266ef2a0[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:19,293 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:19,293 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,298 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,298 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:19,298 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:19,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,299 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:19,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,299 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:19,299 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:19,299 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:19,299 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:19,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,299 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:19,299 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:19,299 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:19,301 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:19,301 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:19,301 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,301 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:19,302 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,301 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:19,302 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:19,302 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:19,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:19,302 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,303 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:19,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:19,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:19,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:19,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:00:19,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:19,308 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 22:00:19,308 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:19,308 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:19,308 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,308 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:19,315 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,315 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,315 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,315 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,315 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,315 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,315 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,316 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:19,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,317 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:19,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,319 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:19,322 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,323 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,323 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,333 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,333 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,334 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,334 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,334 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,334 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,334 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,334 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,334 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:19,336 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:19,334 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,336 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,336 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:19,336 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,336 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:19,336 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,336 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,336 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,337 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,337 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,337 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,337 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,436 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:19,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:19,470 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:19,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:19,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,474 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:19,474 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,475 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 22:00:19,475 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 22:00:19,476 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,476 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,477 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,477 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,477 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,477 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,477 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,477 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:19,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:19,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:19,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:19,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,478 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,479 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:19,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,479 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:19,479 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,480 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:19,480 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:19,480 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:19,480 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,480 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,480 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:19,480 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:19,480 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:19,480 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:19,480 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:19,480 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,480 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,481 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:19,480 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. started... 2023-02-13 22:00:19,482 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:19,482 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:19,482 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:19,482 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:19,482 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 091ee0643bbb09cdba3c6e8d5b8e5835 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:19,482 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:19,499 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,499 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,505 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,505 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:19,505 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,505 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:19,510 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/c764b1e165dd44ada9bd62739b51c15c 2023-02-13 22:00:19,554 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/d2a242eacaa04b88b5180766bae6c8e6 2023-02-13 22:00:19,566 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/c764b1e165dd44ada9bd62739b51c15c as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c 2023-02-13 22:00:19,576 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:00:19,578 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/d2a242eacaa04b88b5180766bae6c8e6 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6 2023-02-13 22:00:19,596 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:00:19,598 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 091ee0643bbb09cdba3c6e8d5b8e5835 in 116ms, sequenceid=32, compaction requested=true 2023-02-13 22:00:19,598 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 091ee0643bbb09cdba3c6e8d5b8e5835: 2023-02-13 22:00:19,598 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:19,598 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:19,599 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:19,599 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,599 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:19,599 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:19,601 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,601 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:19,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:19,601 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,601 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:19,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:19,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:19,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:19,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 22:00:19,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a2c41c7[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:19,604 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:19,604 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,605 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,605 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,605 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,605 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:19,605 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,606 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:19,605 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:19,606 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:19,606 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:19,605 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:19,606 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:19,606 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:19,606 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:19,606 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:19,608 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:19,608 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:19,608 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,608 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:19,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,608 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,608 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,609 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:19,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:19,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:19,609 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,610 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:19,611 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:19,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:00:19,616 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:19,616 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 22:00:19,616 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:19,616 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:19,616 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,616 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:19,623 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,623 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,623 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,623 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,623 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,623 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,623 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:19,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:19,626 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,627 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:19,627 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,639 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:19,640 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,641 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,647 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,647 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,647 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,647 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,647 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,647 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,647 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,648 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,648 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,648 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,648 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,647 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,648 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,648 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,648 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,648 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,647 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,648 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,648 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:19,648 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:19,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,650 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:19,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,650 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:19,750 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:19,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:19,779 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:19,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:19,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,785 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:19,785 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,785 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 22:00:19,785 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 22:00:19,786 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,786 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,787 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,788 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,788 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,788 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,788 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,788 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:19,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:19,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,789 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,790 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:19,790 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,790 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:19,790 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,791 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:19,791 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:19,791 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,792 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:19,792 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,791 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:19,792 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:19,792 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:19,792 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:19,791 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,793 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:19,793 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:19,795 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:19,797 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:19,797 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:19,797 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. started... 2023-02-13 22:00:19,796 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:19,797 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:19,797 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:19,797 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 091ee0643bbb09cdba3c6e8d5b8e5835 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:19,798 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,799 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,799 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:19,800 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,800 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,800 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:19,841 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/783497dac1754c30a3a708f06bb970ed 2023-02-13 22:00:19,901 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/d6af5e7ec62942d29becaf9b4d02aa91 2023-02-13 22:00:19,912 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/783497dac1754c30a3a708f06bb970ed as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed 2023-02-13 22:00:19,923 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:00:19,924 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/d6af5e7ec62942d29becaf9b4d02aa91 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91 2023-02-13 22:00:19,934 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:00:19,935 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 091ee0643bbb09cdba3c6e8d5b8e5835 in 138ms, sequenceid=45, compaction requested=true 2023-02-13 22:00:19,935 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 091ee0643bbb09cdba3c6e8d5b8e5835: 2023-02-13 22:00:19,935 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:19,936 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:19,936 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:19,936 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,936 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:19,937 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:19,939 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:19,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:19,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:19,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:19,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,941 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,941 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,942 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:19,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:19,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 22:00:19,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18c0cfa0[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:19,942 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:19,942 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,943 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,943 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,943 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,943 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:19,943 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:19,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,943 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:19,944 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:19,944 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:19,944 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:19,943 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:19,944 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,944 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:19,944 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:19,944 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:19,945 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:19,945 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,945 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:19,946 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:19,946 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,946 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:19,947 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,947 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:19,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:19,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:19,948 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,948 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:19,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:19,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:19,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:19,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:00:19,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:19,952 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 22:00:19,952 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:19,952 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:19,952 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,952 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:19,954 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,954 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,954 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,954 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,954 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:19,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,954 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:19,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,954 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:19,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,956 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,956 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:19,956 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:19,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,958 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,958 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,959 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,961 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,961 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,961 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,961 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,961 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:19,961 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,961 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,961 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,962 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:19,962 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:19,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:19,961 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:19,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,962 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:19,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:19,961 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,963 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:19,963 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:19,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,963 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:19,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:19,963 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:19,963 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,963 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:19,964 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:19,964 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:19,964 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,063 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:20,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:20,097 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:20,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:20,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,101 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:20,101 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:20,102 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 22:00:20,102 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 22:00:20,102 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,102 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,103 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,103 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,103 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,103 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,104 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,104 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:20,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,105 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,105 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:20,105 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:20,105 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:20,105 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,106 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:20,106 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:20,106 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:20,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,106 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:20,106 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:20,106 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:20,106 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:20,107 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. started... 2023-02-13 22:00:20,107 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:20,107 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:20,106 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:20,107 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:20,109 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:20,109 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:20,109 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:20,107 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 091ee0643bbb09cdba3c6e8d5b8e5835 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:20,115 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,115 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,115 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,115 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:20,115 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,115 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:20,145 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/37f0d725dcdf422f8a95b1c2e80a2de6 2023-02-13 22:00:20,184 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/92d1817df1b04a6ba7eb2e41d66725d6 2023-02-13 22:00:20,192 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/37f0d725dcdf422f8a95b1c2e80a2de6 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/37f0d725dcdf422f8a95b1c2e80a2de6 2023-02-13 22:00:20,201 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/37f0d725dcdf422f8a95b1c2e80a2de6, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:00:20,202 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/92d1817df1b04a6ba7eb2e41d66725d6 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/92d1817df1b04a6ba7eb2e41d66725d6 2023-02-13 22:00:20,212 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/92d1817df1b04a6ba7eb2e41d66725d6, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:00:20,213 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 091ee0643bbb09cdba3c6e8d5b8e5835 in 106ms, sequenceid=58, compaction requested=true 2023-02-13 22:00:20,213 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 091ee0643bbb09cdba3c6e8d5b8e5835: 2023-02-13 22:00:20,213 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:20,213 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:20,214 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:20,215 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,215 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:20,215 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:20,216 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,216 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:20,217 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,217 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:20,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:20,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:20,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:20,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 22:00:20,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f7f1b4f[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:20,219 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:20,219 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,220 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,220 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,220 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,221 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,221 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,221 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:20,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,221 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:20,221 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:20,221 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,221 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:20,221 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:20,221 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:20,221 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:20,221 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:20,222 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:20,222 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,222 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:20,222 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:20,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:20,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:20,223 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:20,223 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:20,223 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:20,223 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:20,223 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:20,223 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:20,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:20,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:00:20,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:20,228 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 22:00:20,228 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:20,228 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:20,228 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,228 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:20,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,238 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,238 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,238 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:20,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:20,238 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:20,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,240 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,240 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:20,240 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,247 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,247 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:20,247 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:20,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,247 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,248 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,248 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,248 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:20,248 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,248 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,248 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:20,248 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,248 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,348 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:20,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:20,380 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:20,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:20,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,382 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:20,382 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:20,383 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 22:00:20,383 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 22:00:20,384 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,384 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,385 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,385 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,385 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,385 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,385 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,385 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:20,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:20,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:20,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:20,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,388 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,388 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:20,388 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:20,388 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 22:00:20,388 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,388 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,388 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:20,388 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:20,388 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,389 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:20,389 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:20,389 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:20,388 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:20,389 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:20,389 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 22:00:20,390 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:20,390 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:20,390 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:20,390 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:20,390 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 22:00:20,391 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:20,391 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:20,391 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:20,391 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. started... 2023-02-13 22:00:20,391 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 091ee0643bbb09cdba3c6e8d5b8e5835 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:20,409 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,412 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,412 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,412 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:20,413 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,413 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/d4ecb3f2cf7d40eca96b28c1f262d05c 2023-02-13 22:00:20,413 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:20,431 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/f1ed29645f6f45debe9bce70dcf23739 2023-02-13 22:00:20,439 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/d4ecb3f2cf7d40eca96b28c1f262d05c as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/d4ecb3f2cf7d40eca96b28c1f262d05c 2023-02-13 22:00:20,446 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/d4ecb3f2cf7d40eca96b28c1f262d05c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:00:20,447 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/f1ed29645f6f45debe9bce70dcf23739 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/f1ed29645f6f45debe9bce70dcf23739 2023-02-13 22:00:20,455 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/f1ed29645f6f45debe9bce70dcf23739, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:00:20,456 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 091ee0643bbb09cdba3c6e8d5b8e5835 in 65ms, sequenceid=71, compaction requested=true 2023-02-13 22:00:20,456 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 091ee0643bbb09cdba3c6e8d5b8e5835: 2023-02-13 22:00:20,456 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:20,456 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:20,457 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:20,457 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,457 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 22:00:20,457 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 22:00:20,458 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,458 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:20,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:20,459 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,459 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:20,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:20,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:20,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:20,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 22:00:20,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ed505f2[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:20,461 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:20,461 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,462 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,462 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,462 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,462 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:20,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,462 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:20,462 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:20,462 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:20,462 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:20,462 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,462 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:20,462 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:20,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,462 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 22:00:20,462 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 22:00:20,462 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:20,463 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:20,463 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:20,463 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:20,464 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:20,464 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 22:00:20,464 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:20,464 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:20,464 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,464 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:20,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:20,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:20,465 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 22:00:20,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:20,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:20,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:20,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:00:20,468 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:20,468 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 22:00:20,468 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:20,468 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:20,469 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,469 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:20,470 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,470 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,470 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:20,470 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,470 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:20,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,470 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,470 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,470 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:20,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,471 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,471 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:20,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,472 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,473 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:20,473 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,479 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,479 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,479 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,479 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,479 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,479 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,479 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:20,479 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,479 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 22:00:20,480 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:20,479 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:20,479 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,480 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:20,479 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:20,481 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:20,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:20,481 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,481 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:20,481 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,482 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:20,482 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:20,482 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,482 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 22:00:20,582 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:20,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:20,612 DEBUG [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,618 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:20,619 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:20,619 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:20,619 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:20,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=091ee0643bbb09cdba3c6e8d5b8e5835, UNASSIGN}] 2023-02-13 22:00:20,627 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=091ee0643bbb09cdba3c6e8d5b8e5835, UNASSIGN 2023-02-13 22:00:20,628 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=091ee0643bbb09cdba3c6e8d5b8e5835, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:20,628 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676325620628"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325620628"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325620628"}]},"ts":"1676325620628"} 2023-02-13 22:00:20,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 091ee0643bbb09cdba3c6e8d5b8e5835, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:00:20,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 091ee0643bbb09cdba3c6e8d5b8e5835, disabling compactions & flushes 2023-02-13 22:00:20,796 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:20,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:20,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. after waiting 0 ms 2023-02-13 22:00:20,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:20,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 091ee0643bbb09cdba3c6e8d5b8e5835 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 22:00:20,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/a2caaf32a2ac4210bbb46aba2e87b4f6 2023-02-13 22:00:20,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/335773dbd3324c9092952003333c388e 2023-02-13 22:00:20,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf1/a2caaf32a2ac4210bbb46aba2e87b4f6 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/a2caaf32a2ac4210bbb46aba2e87b4f6 2023-02-13 22:00:20,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/a2caaf32a2ac4210bbb46aba2e87b4f6, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:00:20,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.tmp/cf2/335773dbd3324c9092952003333c388e as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/335773dbd3324c9092952003333c388e 2023-02-13 22:00:20,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/335773dbd3324c9092952003333c388e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:00:20,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 091ee0643bbb09cdba3c6e8d5b8e5835 in 62ms, sequenceid=82, compaction requested=true 2023-02-13 22:00:20,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 22:00:20,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:20,880 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. 2023-02-13 22:00:20,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 091ee0643bbb09cdba3c6e8d5b8e5835: Waiting for close lock at 1676325620796Running coprocessor pre-close hooks at 1676325620796Disabling compacts and flushes for region at 1676325620796Disabling writes for close at 1676325620796Obtaining lock to block concurrent updates at 1676325620797 (+1 ms)Preparing flush snapshotting stores in 091ee0643bbb09cdba3c6e8d5b8e5835 at 1676325620797Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676325620797Flushing stores of testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. at 1676325620799 (+2 ms)Flushing 091ee0643bbb09cdba3c6e8d5b8e5835/cf1: creating writer at 1676325620800 (+1 ms)Flushing 091ee0643bbb09cdba3c6e8d5b8e5835/cf1: appending metadata at 1676325620803 (+3 ms)Flushing 091ee0643bbb09cdba3c6e8d5b8e5835/cf1: closing flushed file at 1676325620803Flushing 091ee0643bbb09cdba3c6e8d5b8e5835/cf2: creating writer at 1676325620823 (+20 ms)Flushing 091ee0643bbb09cdba3c6e8d5b8e5835/cf2: appending metadata at 1676325620825 (+2 ms)Flushing 091ee0643bbb09cdba3c6e8d5b8e5835/cf2: closing flushed file at 1676325620825Flushing 091ee0643bbb09cdba3c6e8d5b8e5835/cf1: reopening flushed file at 1676325620842 (+17 ms)Flushing 091ee0643bbb09cdba3c6e8d5b8e5835/cf2: reopening flushed file at 1676325620851 (+9 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 091ee0643bbb09cdba3c6e8d5b8e5835 in 62ms, sequenceid=82, compaction requested=true at 1676325620858 (+7 ms)Writing region close event to WAL at 1676325620873 (+15 ms)Running coprocessor post-close hooks at 1676325620879 (+6 ms)Closed at 1676325620880 (+1 ms) 2023-02-13 22:00:20,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=091ee0643bbb09cdba3c6e8d5b8e5835, regionState=CLOSED 2023-02-13 22:00:20,885 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676325620884"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325620884"}]},"ts":"1676325620884"} 2023-02-13 22:00:20,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 22:00:20,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 091ee0643bbb09cdba3c6e8d5b8e5835, server=jenkins-hbase20.apache.org,42047,1676325615467 in 256 msec 2023-02-13 22:00:20,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-13 22:00:20,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=091ee0643bbb09cdba3c6e8d5b8e5835, UNASSIGN in 263 msec 2023-02-13 22:00:20,910 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=091ee0643bbb09cdba3c6e8d5b8e5835, threads=14 2023-02-13 22:00:20,911 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/335773dbd3324c9092952003333c388e for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,911 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/92d1817df1b04a6ba7eb2e41d66725d6 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,911 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,911 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,911 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,911 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,912 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/f1ed29645f6f45debe9bce70dcf23739 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,912 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,912 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,913 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/37f0d725dcdf422f8a95b1c2e80a2de6 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,913 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,913 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/a2caaf32a2ac4210bbb46aba2e87b4f6 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,913 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,913 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/d4ecb3f2cf7d40eca96b28c1f262d05c for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,933 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/f1ed29645f6f45debe9bce70dcf23739, top=true 2023-02-13 22:00:20,938 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/d4ecb3f2cf7d40eca96b28c1f262d05c, top=true 2023-02-13 22:00:20,938 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/37f0d725dcdf422f8a95b1c2e80a2de6, top=true 2023-02-13 22:00:20,939 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/335773dbd3324c9092952003333c388e, top=true 2023-02-13 22:00:20,947 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/92d1817df1b04a6ba7eb2e41d66725d6, top=true 2023-02-13 22:00:20,957 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/a2caaf32a2ac4210bbb46aba2e87b4f6, top=true 2023-02-13 22:00:20,959 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.splits/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-f1ed29645f6f45debe9bce70dcf23739 for child: a8a1dfec1256014dc4fdfc9c8ef9862d, parent: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,959 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/f1ed29645f6f45debe9bce70dcf23739 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,991 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.splits/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-d4ecb3f2cf7d40eca96b28c1f262d05c for child: a8a1dfec1256014dc4fdfc9c8ef9862d, parent: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,998 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.splits/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-335773dbd3324c9092952003333c388e for child: a8a1dfec1256014dc4fdfc9c8ef9862d, parent: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,998 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.splits/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-37f0d725dcdf422f8a95b1c2e80a2de6 for child: a8a1dfec1256014dc4fdfc9c8ef9862d, parent: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,999 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.splits/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-92d1817df1b04a6ba7eb2e41d66725d6 for child: a8a1dfec1256014dc4fdfc9c8ef9862d, parent: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,999 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/335773dbd3324c9092952003333c388e for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:20,999 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/d4ecb3f2cf7d40eca96b28c1f262d05c for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,007 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/92d1817df1b04a6ba7eb2e41d66725d6 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,001 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/37f0d725dcdf422f8a95b1c2e80a2de6 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,017 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/.splits/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-a2caaf32a2ac4210bbb46aba2e87b4f6 for child: a8a1dfec1256014dc4fdfc9c8ef9862d, parent: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,017 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/a2caaf32a2ac4210bbb46aba2e87b4f6 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,078 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,090 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,097 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,098 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,100 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,439 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91 for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,483 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,484 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc for region: 091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:21,484 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 091ee0643bbb09cdba3c6e8d5b8e5835 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-13 22:00:21,539 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:00:21,541 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:00:21,544 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676325621543"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676325621543"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676325621543"}]},"ts":"1676325621543"} 2023-02-13 22:00:21,544 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676325621543"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325621543"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325621543"}]},"ts":"1676325621543"} 2023-02-13 22:00:21,544 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676325621543"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325621543"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325621543"}]},"ts":"1676325621543"} 2023-02-13 22:00:21,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d4dc0951c7044c39c81d72b688a3b019, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a8a1dfec1256014dc4fdfc9c8ef9862d, ASSIGN}] 2023-02-13 22:00:21,586 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a8a1dfec1256014dc4fdfc9c8ef9862d, ASSIGN 2023-02-13 22:00:21,586 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a8a1dfec1256014dc4fdfc9c8ef9862d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,42047,1676325615467; forceNewPlan=false, retain=false 2023-02-13 22:00:21,587 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d4dc0951c7044c39c81d72b688a3b019, ASSIGN 2023-02-13 22:00:21,588 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d4dc0951c7044c39c81d72b688a3b019, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,42047,1676325615467; forceNewPlan=false, retain=false 2023-02-13 22:00:21,737 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 22:00:21,741 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=d4dc0951c7044c39c81d72b688a3b019, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:21,741 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676325621740"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325621740"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325621740"}]},"ts":"1676325621740"} 2023-02-13 22:00:21,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure d4dc0951c7044c39c81d72b688a3b019, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:00:21,753 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=a8a1dfec1256014dc4fdfc9c8ef9862d, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:21,753 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676325621753"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325621753"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325621753"}]},"ts":"1676325621753"} 2023-02-13 22:00:21,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure a8a1dfec1256014dc4fdfc9c8ef9862d, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:00:21,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:21,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d4dc0951c7044c39c81d72b688a3b019, NAME => 'testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.', STARTKEY => '', ENDKEY => '26'} 2023-02-13 22:00:21,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:21,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:21,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:21,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:21,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:21,914 INFO [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:21,915 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1 2023-02-13 22:00:21,915 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1 2023-02-13 22:00:21,916 INFO [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4dc0951c7044c39c81d72b688a3b019 columnFamilyName cf1 2023-02-13 22:00:21,948 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde-bottom 2023-02-13 22:00:21,956 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed-bottom 2023-02-13 22:00:21,964 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558-bottom 2023-02-13 22:00:21,972 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c-bottom 2023-02-13 22:00:21,972 INFO [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(310): Store=d4dc0951c7044c39c81d72b688a3b019/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:21,972 INFO [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:21,974 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2 2023-02-13 22:00:21,974 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2 2023-02-13 22:00:21,974 INFO [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4dc0951c7044c39c81d72b688a3b019 columnFamilyName cf2 2023-02-13 22:00:21,989 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc-bottom 2023-02-13 22:00:21,996 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e-bottom 2023-02-13 22:00:22,003 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6-bottom 2023-02-13 22:00:22,010 DEBUG [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91-bottom 2023-02-13 22:00:22,011 INFO [StoreOpener-d4dc0951c7044c39c81d72b688a3b019-1] regionserver.HStore(310): Store=d4dc0951c7044c39c81d72b688a3b019/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:22,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:22,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:22,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:22,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:22,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened d4dc0951c7044c39c81d72b688a3b019; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63014586, jitterRate=-0.061009496450424194}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:22,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for d4dc0951c7044c39c81d72b688a3b019: Running coprocessor pre-open hook at 1676325621912Writing region info on filesystem at 1676325621912Initializing all the Stores at 1676325621913 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325621913Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325621913Cleaning up temporary data from old regions at 1676325622014 (+101 ms)Cleaning up detritus from prior splits at 1676325622015 (+1 ms)Running coprocessor post-open hooks at 1676325622018 (+3 ms)Region opened successfully at 1676325622018 2023-02-13 22:00:22,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019., pid=17, masterSystemTime=1676325621901 2023-02-13 22:00:22,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:22,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:22,022 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 22:00:22,022 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 22:00:22,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:22,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:22,024 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:22,024 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1912): d4dc0951c7044c39c81d72b688a3b019/cf1 is initiating minor compaction (all files) 2023-02-13 22:00:22,024 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:22,025 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d4dc0951c7044c39c81d72b688a3b019/cf1 in testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:22,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,025 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1912): d4dc0951c7044c39c81d72b688a3b019/cf2 is initiating minor compaction (all files) 2023-02-13 22:00:22,025 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegion(2258): Starting compaction of d4dc0951c7044c39c81d72b688a3b019/cf2 in testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:22,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a8a1dfec1256014dc4fdfc9c8ef9862d, NAME => 'testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.', STARTKEY => '26', ENDKEY => ''} 2023-02-13 22:00:22,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:22,025 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91-bottom] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/.tmp, totalSize=19.9 K 2023-02-13 22:00:22,025 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed-bottom] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/.tmp, totalSize=19.9 K 2023-02-13 22:00:22,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:22,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,027 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=d4dc0951c7044c39c81d72b688a3b019, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:22,028 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676325622027"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325622027"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325622027"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325622027"}]},"ts":"1676325622027"} 2023-02-13 22:00:22,028 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325618706 2023-02-13 22:00:22,029 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325618706 2023-02-13 22:00:22,030 INFO [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,031 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325619134 2023-02-13 22:00:22,031 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325619134 2023-02-13 22:00:22,032 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325619439 2023-02-13 22:00:22,032 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1 2023-02-13 22:00:22,032 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325619753 2023-02-13 22:00:22,032 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325619439 2023-02-13 22:00:22,032 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1 2023-02-13 22:00:22,033 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325619753 2023-02-13 22:00:22,033 INFO [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8a1dfec1256014dc4fdfc9c8ef9862d columnFamilyName cf1 2023-02-13 22:00:22,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-13 22:00:22,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure d4dc0951c7044c39c81d72b688a3b019, server=jenkins-hbase20.apache.org,42047,1676325615467 in 285 msec 2023-02-13 22:00:22,039 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d4dc0951c7044c39c81d72b688a3b019, ASSIGN in 453 msec 2023-02-13 22:00:22,052 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] throttle.PressureAwareThroughputController(145): d4dc0951c7044c39c81d72b688a3b019#cf1#compaction#14 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:22,067 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde-top 2023-02-13 22:00:22,071 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] throttle.PressureAwareThroughputController(145): d4dc0951c7044c39c81d72b688a3b019#cf2#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:22,074 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed-top 2023-02-13 22:00:22,089 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558-top 2023-02-13 22:00:22,098 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/.tmp/cf1/8b963b65c516427c9c06702cb83dccdf as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/8b963b65c516427c9c06702cb83dccdf 2023-02-13 22:00:22,099 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c-top 2023-02-13 22:00:22,105 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/.tmp/cf2/02af2cdcf4fc48c283e58a49fc917069 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/02af2cdcf4fc48c283e58a49fc917069 2023-02-13 22:00:22,106 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-37f0d725dcdf422f8a95b1c2e80a2de6 2023-02-13 22:00:22,112 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-a2caaf32a2ac4210bbb46aba2e87b4f6 2023-02-13 22:00:22,117 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d4dc0951c7044c39c81d72b688a3b019/cf2 of d4dc0951c7044c39c81d72b688a3b019 into 02af2cdcf4fc48c283e58a49fc917069(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:22,117 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for d4dc0951c7044c39c81d72b688a3b019: 2023-02-13 22:00:22,117 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019., storeName=d4dc0951c7044c39c81d72b688a3b019/cf2, priority=12, startTime=1676325622021; duration=0sec 2023-02-13 22:00:22,117 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d4dc0951c7044c39c81d72b688a3b019/cf1 of d4dc0951c7044c39c81d72b688a3b019 into 8b963b65c516427c9c06702cb83dccdf(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:22,117 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d4dc0951c7044c39c81d72b688a3b019: 2023-02-13 22:00:22,117 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019., storeName=d4dc0951c7044c39c81d72b688a3b019/cf1, priority=12, startTime=1676325622020; duration=0sec 2023-02-13 22:00:22,118 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:22,118 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:22,120 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-d4ecb3f2cf7d40eca96b28c1f262d05c 2023-02-13 22:00:22,120 INFO [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(310): Store=a8a1dfec1256014dc4fdfc9c8ef9862d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:22,120 INFO [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,122 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2 2023-02-13 22:00:22,122 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2 2023-02-13 22:00:22,122 INFO [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8a1dfec1256014dc4fdfc9c8ef9862d columnFamilyName cf2 2023-02-13 22:00:22,145 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc-top 2023-02-13 22:00:22,151 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e-top 2023-02-13 22:00:22,156 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6-top 2023-02-13 22:00:22,163 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91-top 2023-02-13 22:00:22,168 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-335773dbd3324c9092952003333c388e 2023-02-13 22:00:22,173 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-92d1817df1b04a6ba7eb2e41d66725d6 2023-02-13 22:00:22,178 DEBUG [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-f1ed29645f6f45debe9bce70dcf23739 2023-02-13 22:00:22,179 INFO [StoreOpener-a8a1dfec1256014dc4fdfc9c8ef9862d-1] regionserver.HStore(310): Store=a8a1dfec1256014dc4fdfc9c8ef9862d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:22,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:22,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened a8a1dfec1256014dc4fdfc9c8ef9862d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69769903, jitterRate=0.03965257108211517}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:22,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for a8a1dfec1256014dc4fdfc9c8ef9862d: Running coprocessor pre-open hook at 1676325622027Writing region info on filesystem at 1676325622027Initializing all the Stores at 1676325622028 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325622029 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325622030 (+1 ms)Cleaning up temporary data from old regions at 1676325622181 (+151 ms)Cleaning up detritus from prior splits at 1676325622182 (+1 ms)Running coprocessor post-open hooks at 1676325622184 (+2 ms)Region opened successfully at 1676325622184 2023-02-13 22:00:22,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d., pid=18, masterSystemTime=1676325621901 2023-02-13 22:00:22,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:22,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:22,186 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:22,186 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:22,186 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,186 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,187 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1912): a8a1dfec1256014dc4fdfc9c8ef9862d/cf1 is initiating minor compaction (all files) 2023-02-13 22:00:22,187 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1912): a8a1dfec1256014dc4fdfc9c8ef9862d/cf2 is initiating minor compaction (all files) 2023-02-13 22:00:22,187 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a8a1dfec1256014dc4fdfc9c8ef9862d/cf1 in testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,187 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a8a1dfec1256014dc4fdfc9c8ef9862d/cf2 in testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,187 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-92d1817df1b04a6ba7eb2e41d66725d6, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-f1ed29645f6f45debe9bce70dcf23739, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-335773dbd3324c9092952003333c388e] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/.tmp, totalSize=34.9 K 2023-02-13 22:00:22,187 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-37f0d725dcdf422f8a95b1c2e80a2de6, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-d4ecb3f2cf7d40eca96b28c1f262d05c, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-a2caaf32a2ac4210bbb46aba2e87b4f6] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/.tmp, totalSize=34.9 K 2023-02-13 22:00:22,188 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676325618706 2023-02-13 22:00:22,188 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676325618706 2023-02-13 22:00:22,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,189 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676325619134 2023-02-13 22:00:22,189 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676325619134 2023-02-13 22:00:22,189 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676325619439 2023-02-13 22:00:22,190 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676325619439 2023-02-13 22:00:22,190 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=a8a1dfec1256014dc4fdfc9c8ef9862d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:22,190 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676325622190"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325622190"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325622190"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325622190"}]},"ts":"1676325622190"} 2023-02-13 22:00:22,191 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676325619753 2023-02-13 22:00:22,191 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676325619753 2023-02-13 22:00:22,191 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-37f0d725dcdf422f8a95b1c2e80a2de6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325620066 2023-02-13 22:00:22,192 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-92d1817df1b04a6ba7eb2e41d66725d6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325620066 2023-02-13 22:00:22,192 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-d4ecb3f2cf7d40eca96b28c1f262d05c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325620352 2023-02-13 22:00:22,192 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-f1ed29645f6f45debe9bce70dcf23739, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325620352 2023-02-13 22:00:22,192 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-a2caaf32a2ac4210bbb46aba2e87b4f6, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325620588 2023-02-13 22:00:22,193 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-335773dbd3324c9092952003333c388e, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325620588 2023-02-13 22:00:22,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-13 22:00:22,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure a8a1dfec1256014dc4fdfc9c8ef9862d, server=jenkins-hbase20.apache.org,42047,1676325615467 in 437 msec 2023-02-13 22:00:22,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-13 22:00:22,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a8a1dfec1256014dc4fdfc9c8ef9862d, ASSIGN in 614 msec 2023-02-13 22:00:22,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d in 1.5900 sec 2023-02-13 22:00:22,219 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] throttle.PressureAwareThroughputController(145): a8a1dfec1256014dc4fdfc9c8ef9862d#cf2#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:22,219 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] throttle.PressureAwareThroughputController(145): a8a1dfec1256014dc4fdfc9c8ef9862d#cf1#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:22,240 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/.tmp/cf2/56a4dbe43b3344778dc416ea04617ccb as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/56a4dbe43b3344778dc416ea04617ccb 2023-02-13 22:00:22,240 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/.tmp/cf1/22365e28e7ea4639a6996f4bfa75aaa1 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/22365e28e7ea4639a6996f4bfa75aaa1 2023-02-13 22:00:22,247 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a8a1dfec1256014dc4fdfc9c8ef9862d/cf1 of a8a1dfec1256014dc4fdfc9c8ef9862d into 22365e28e7ea4639a6996f4bfa75aaa1(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:22,247 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a8a1dfec1256014dc4fdfc9c8ef9862d/cf2 of a8a1dfec1256014dc4fdfc9c8ef9862d into 56a4dbe43b3344778dc416ea04617ccb(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:22,247 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a8a1dfec1256014dc4fdfc9c8ef9862d: 2023-02-13 22:00:22,247 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a8a1dfec1256014dc4fdfc9c8ef9862d: 2023-02-13 22:00:22,247 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d., storeName=a8a1dfec1256014dc4fdfc9c8ef9862d/cf1, priority=9, startTime=1676325622185; duration=0sec 2023-02-13 22:00:22,247 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d., storeName=a8a1dfec1256014dc4fdfc9c8ef9862d/cf2, priority=9, startTime=1676325622186; duration=0sec 2023-02-13 22:00:22,247 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:22,247 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:22,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42047] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 148.251.75.209:45782 deadline: 1676325682371, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835. is not online on jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:22,497 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 22:00:22,510 INFO [Listener at localhost.localdomain/42295] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 22:00:22,513 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 22:00:22,542 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:22,550 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-13 22:00:22,554 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitTableRegionUnevenDaughter 2023-02-13 22:00:22,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:22,565 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325622564"}]},"ts":"1676325622564"} 2023-02-13 22:00:22,566 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-13 22:00:22,568 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-13 22:00:22,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d4dc0951c7044c39c81d72b688a3b019, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a8a1dfec1256014dc4fdfc9c8ef9862d, UNASSIGN}] 2023-02-13 22:00:22,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 22:00:22,571 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a8a1dfec1256014dc4fdfc9c8ef9862d, UNASSIGN 2023-02-13 22:00:22,572 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=a8a1dfec1256014dc4fdfc9c8ef9862d, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:22,573 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676325622572"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325622572"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325622572"}]},"ts":"1676325622572"} 2023-02-13 22:00:22,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure a8a1dfec1256014dc4fdfc9c8ef9862d, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:00:22,579 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d4dc0951c7044c39c81d72b688a3b019, UNASSIGN 2023-02-13 22:00:22,580 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=d4dc0951c7044c39c81d72b688a3b019, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:22,581 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676325622580"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325622580"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325622580"}]},"ts":"1676325622580"} 2023-02-13 22:00:22,583 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure d4dc0951c7044c39c81d72b688a3b019, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:00:22,640 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 22:00:22,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 22:00:22,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:22,728 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing d4dc0951c7044c39c81d72b688a3b019, disabling compactions & flushes 2023-02-13 22:00:22,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:22,728 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:22,728 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. after waiting 0 ms 2023-02-13 22:00:22,728 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:22,735 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed-bottom] to archive 2023-02-13 22:00:22,737 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:22,741 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 22:00:22,741 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 22:00:22,746 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,748 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,750 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,753 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,782 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91-bottom] to archive 2023-02-13 22:00:22,783 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:22,785 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,789 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,790 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,792 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,799 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 22:00:22,800 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:22,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019. 2023-02-13 22:00:22,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for d4dc0951c7044c39c81d72b688a3b019: Waiting for close lock at 1676325622728Running coprocessor pre-close hooks at 1676325622728Disabling compacts and flushes for region at 1676325622728Disabling writes for close at 1676325622728Writing region close event to WAL at 1676325622793 (+65 ms)Running coprocessor post-close hooks at 1676325622800 (+7 ms)Closed at 1676325622801 (+1 ms) 2023-02-13 22:00:22,803 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:22,803 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing a8a1dfec1256014dc4fdfc9c8ef9862d, disabling compactions & flushes 2023-02-13 22:00:22,803 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. after waiting 0 ms 2023-02-13 22:00:22,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,804 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=d4dc0951c7044c39c81d72b688a3b019, regionState=CLOSED 2023-02-13 22:00:22,804 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676325622804"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325622804"}]},"ts":"1676325622804"} 2023-02-13 22:00:22,805 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-37f0d725dcdf422f8a95b1c2e80a2de6, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-d4ecb3f2cf7d40eca96b28c1f262d05c, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-a2caaf32a2ac4210bbb46aba2e87b4f6] to archive 2023-02-13 22:00:22,806 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:22,809 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/9bca9e48275e41d39aa4b5b565d7e558.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,811 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/1b8a523c71b74209b082af1a0870ccde.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-13 22:00:22,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure d4dc0951c7044c39c81d72b688a3b019, server=jenkins-hbase20.apache.org,42047,1676325615467 in 224 msec 2023-02-13 22:00:22,813 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/c764b1e165dd44ada9bd62739b51c15c.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d4dc0951c7044c39c81d72b688a3b019, UNASSIGN in 241 msec 2023-02-13 22:00:22,815 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/783497dac1754c30a3a708f06bb970ed.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,817 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-37f0d725dcdf422f8a95b1c2e80a2de6 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-37f0d725dcdf422f8a95b1c2e80a2de6 2023-02-13 22:00:22,819 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-d4ecb3f2cf7d40eca96b28c1f262d05c to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-d4ecb3f2cf7d40eca96b28c1f262d05c 2023-02-13 22:00:22,821 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-a2caaf32a2ac4210bbb46aba2e87b4f6 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-a2caaf32a2ac4210bbb46aba2e87b4f6 2023-02-13 22:00:22,823 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-92d1817df1b04a6ba7eb2e41d66725d6, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-f1ed29645f6f45debe9bce70dcf23739, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-335773dbd3324c9092952003333c388e] to archive 2023-02-13 22:00:22,824 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:22,827 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/a3798dcfb8f547bbafdd4bf70501a8fc.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,828 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/ce34be6674484090a2762ff472da269e.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,830 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/d2a242eacaa04b88b5180766bae6c8e6.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,832 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/d6af5e7ec62942d29becaf9b4d02aa91.091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,833 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-92d1817df1b04a6ba7eb2e41d66725d6 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-92d1817df1b04a6ba7eb2e41d66725d6 2023-02-13 22:00:22,834 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-f1ed29645f6f45debe9bce70dcf23739 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-f1ed29645f6f45debe9bce70dcf23739 2023-02-13 22:00:22,836 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-335773dbd3324c9092952003333c388e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/testSplitTableRegionUnevenDaughter=091ee0643bbb09cdba3c6e8d5b8e5835-335773dbd3324c9092952003333c388e 2023-02-13 22:00:22,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 22:00:22,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:22,846 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d. 2023-02-13 22:00:22,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for a8a1dfec1256014dc4fdfc9c8ef9862d: Waiting for close lock at 1676325622803Running coprocessor pre-close hooks at 1676325622803Disabling compacts and flushes for region at 1676325622803Disabling writes for close at 1676325622804 (+1 ms)Writing region close event to WAL at 1676325622837 (+33 ms)Running coprocessor post-close hooks at 1676325622845 (+8 ms)Closed at 1676325622846 (+1 ms) 2023-02-13 22:00:22,849 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,849 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=a8a1dfec1256014dc4fdfc9c8ef9862d, regionState=CLOSED 2023-02-13 22:00:22,850 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676325622849"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325622849"}]},"ts":"1676325622849"} 2023-02-13 22:00:22,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-13 22:00:22,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure a8a1dfec1256014dc4fdfc9c8ef9862d, server=jenkins-hbase20.apache.org,42047,1676325615467 in 278 msec 2023-02-13 22:00:22,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-02-13 22:00:22,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a8a1dfec1256014dc4fdfc9c8ef9862d, UNASSIGN in 286 msec 2023-02-13 22:00:22,860 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325622860"}]},"ts":"1676325622860"} 2023-02-13 22:00:22,862 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-13 22:00:22,863 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-13 22:00:22,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 309 msec 2023-02-13 22:00:22,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 22:00:22,876 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-13 22:00:22,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitTableRegionUnevenDaughter 2023-02-13 22:00:22,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:22,889 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:22,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:22,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 22:00:22,899 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,899 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,899 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:22,904 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/recovered.edits] 2023-02-13 22:00:22,904 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/recovered.edits] 2023-02-13 22:00:22,904 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/recovered.edits] 2023-02-13 22:00:22,907 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/22365e28e7ea4639a6996f4bfa75aaa1 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf1/22365e28e7ea4639a6996f4bfa75aaa1 2023-02-13 22:00:22,908 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/8b963b65c516427c9c06702cb83dccdf to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf1/8b963b65c516427c9c06702cb83dccdf 2023-02-13 22:00:22,910 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/56a4dbe43b3344778dc416ea04617ccb to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/cf2/56a4dbe43b3344778dc416ea04617ccb 2023-02-13 22:00:22,911 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/02af2cdcf4fc48c283e58a49fc917069 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/cf2/02af2cdcf4fc48c283e58a49fc917069 2023-02-13 22:00:22,914 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/1b8a523c71b74209b082af1a0870ccde 2023-02-13 22:00:22,915 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/37f0d725dcdf422f8a95b1c2e80a2de6 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/37f0d725dcdf422f8a95b1c2e80a2de6 2023-02-13 22:00:22,916 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/recovered.edits/90.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019/recovered.edits/90.seqid 2023-02-13 22:00:22,916 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/recovered.edits/90.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d/recovered.edits/90.seqid 2023-02-13 22:00:22,916 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/d4dc0951c7044c39c81d72b688a3b019 2023-02-13 22:00:22,917 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:22,917 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/783497dac1754c30a3a708f06bb970ed 2023-02-13 22:00:22,919 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/9bca9e48275e41d39aa4b5b565d7e558 2023-02-13 22:00:22,920 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/a2caaf32a2ac4210bbb46aba2e87b4f6 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/a2caaf32a2ac4210bbb46aba2e87b4f6 2023-02-13 22:00:22,922 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/c764b1e165dd44ada9bd62739b51c15c 2023-02-13 22:00:22,923 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/d4ecb3f2cf7d40eca96b28c1f262d05c to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf1/d4ecb3f2cf7d40eca96b28c1f262d05c 2023-02-13 22:00:22,926 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/335773dbd3324c9092952003333c388e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/335773dbd3324c9092952003333c388e 2023-02-13 22:00:22,928 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/92d1817df1b04a6ba7eb2e41d66725d6 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/92d1817df1b04a6ba7eb2e41d66725d6 2023-02-13 22:00:22,930 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/a3798dcfb8f547bbafdd4bf70501a8fc 2023-02-13 22:00:22,931 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/ce34be6674484090a2762ff472da269e 2023-02-13 22:00:22,933 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d2a242eacaa04b88b5180766bae6c8e6 2023-02-13 22:00:22,934 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/d6af5e7ec62942d29becaf9b4d02aa91 2023-02-13 22:00:22,936 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/f1ed29645f6f45debe9bce70dcf23739 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/cf2/f1ed29645f6f45debe9bce70dcf23739 2023-02-13 22:00:22,940 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/recovered.edits/85.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835/recovered.edits/85.seqid 2023-02-13 22:00:22,941 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionUnevenDaughter/091ee0643bbb09cdba3c6e8d5b8e5835 2023-02-13 22:00:22,941 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 22:00:22,945 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:22,967 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-13 22:00:22,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 22:00:23,014 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-13 22:00:23,016 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:23,016 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-13 22:00:23,017 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325623016"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:23,017 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325623016"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:23,017 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325623016"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:23,020 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 22:00:23,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 091ee0643bbb09cdba3c6e8d5b8e5835, NAME => 'testSplitTableRegionUnevenDaughter,,1676325618198.091ee0643bbb09cdba3c6e8d5b8e5835.', STARTKEY => '', ENDKEY => ''}, {ENCODED => d4dc0951c7044c39c81d72b688a3b019, NAME => 'testSplitTableRegionUnevenDaughter,,1676325620612.d4dc0951c7044c39c81d72b688a3b019.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => a8a1dfec1256014dc4fdfc9c8ef9862d, NAME => 'testSplitTableRegionUnevenDaughter,26,1676325620612.a8a1dfec1256014dc4fdfc9c8ef9862d.', STARTKEY => '26', ENDKEY => ''}] 2023-02-13 22:00:23,020 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-13 22:00:23,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676325623020"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:23,022 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-13 22:00:23,025 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:23,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 143 msec 2023-02-13 22:00:23,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 22:00:23,196 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-13 22:00:23,215 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=388 (was 356) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:53440 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:53574 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54320 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54388 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1357b8ff-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54344 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54418 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54442 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54360 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:53500 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54402 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54370 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54458 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1357b8ff-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1357b8ff-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:53514 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1357b8ff-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54178 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase20:42047-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54456 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_510700930_15 at /127.0.0.1:54084 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=835 (was 691) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=282 (was 271) - SystemLoadAverage LEAK? -, ProcessCount=164 (was 164), AvailableMemoryMB=7557 (was 7829) 2023-02-13 22:00:23,233 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=388, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=282, ProcessCount=164, AvailableMemoryMB=7556 2023-02-13 22:00:23,234 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:23,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:00:23,238 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-13 22:00:23,240 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:23,242 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:00:23,243 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:00:23,246 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:23,246 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426 empty. 2023-02-13 22:00:23,247 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:23,248 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 22:00:23,261 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:23,262 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4fff278c72d1992f129cd33a6d722426, NAME => 'testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:00:23,273 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:23,274 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 4fff278c72d1992f129cd33a6d722426, disabling compactions & flushes 2023-02-13 22:00:23,274 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:23,274 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:23,274 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. after waiting 0 ms 2023-02-13 22:00:23,274 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:23,274 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:23,274 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 4fff278c72d1992f129cd33a6d722426: Waiting for close lock at 1676325623274Disabling compacts and flushes for region at 1676325623274Disabling writes for close at 1676325623274Writing region close event to WAL at 1676325623274Closed at 1676325623274 2023-02-13 22:00:23,276 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:00:23,277 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325623277"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325623277"}]},"ts":"1676325623277"} 2023-02-13 22:00:23,279 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:00:23,280 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:00:23,281 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325623280"}]},"ts":"1676325623280"} 2023-02-13 22:00:23,282 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-13 22:00:23,284 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:23,285 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:23,285 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:23,285 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:23,285 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:23,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, ASSIGN}] 2023-02-13 22:00:23,287 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, ASSIGN 2023-02-13 22:00:23,288 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:00:23,438 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:23,439 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=4fff278c72d1992f129cd33a6d722426, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,440 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325623439"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325623439"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325623439"}]},"ts":"1676325623439"} 2023-02-13 22:00:23,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:23,596 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,596 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 22:00:23,603 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:49494, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 22:00:23,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:23,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4fff278c72d1992f129cd33a6d722426, NAME => 'testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:23,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:23,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:23,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:23,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:23,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:23,609 INFO [StoreOpener-4fff278c72d1992f129cd33a6d722426-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:23,610 DEBUG [StoreOpener-4fff278c72d1992f129cd33a6d722426-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1 2023-02-13 22:00:23,610 DEBUG [StoreOpener-4fff278c72d1992f129cd33a6d722426-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1 2023-02-13 22:00:23,611 INFO [StoreOpener-4fff278c72d1992f129cd33a6d722426-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fff278c72d1992f129cd33a6d722426 columnFamilyName cf1 2023-02-13 22:00:23,611 INFO [StoreOpener-4fff278c72d1992f129cd33a6d722426-1] regionserver.HStore(310): Store=4fff278c72d1992f129cd33a6d722426/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:23,611 INFO [StoreOpener-4fff278c72d1992f129cd33a6d722426-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:23,613 DEBUG [StoreOpener-4fff278c72d1992f129cd33a6d722426-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2 2023-02-13 22:00:23,614 DEBUG [StoreOpener-4fff278c72d1992f129cd33a6d722426-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2 2023-02-13 22:00:23,615 INFO [StoreOpener-4fff278c72d1992f129cd33a6d722426-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fff278c72d1992f129cd33a6d722426 columnFamilyName cf2 2023-02-13 22:00:23,616 INFO [StoreOpener-4fff278c72d1992f129cd33a6d722426-1] regionserver.HStore(310): Store=4fff278c72d1992f129cd33a6d722426/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:23,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:23,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:23,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:23,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:23,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:23,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 4fff278c72d1992f129cd33a6d722426; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64489675, jitterRate=-0.03902895748615265}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:23,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 4fff278c72d1992f129cd33a6d722426: Running coprocessor pre-open hook at 1676325623607Writing region info on filesystem at 1676325623607Initializing all the Stores at 1676325623608 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325623608Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325623608Cleaning up temporary data from old regions at 1676325623618 (+10 ms)Cleaning up detritus from prior splits at 1676325623619 (+1 ms)Running coprocessor post-open hooks at 1676325623625 (+6 ms)Region opened successfully at 1676325623625 2023-02-13 22:00:23,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426., pid=27, masterSystemTime=1676325623596 2023-02-13 22:00:23,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:23,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:23,631 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=4fff278c72d1992f129cd33a6d722426, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,631 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325623631"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325623631"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325623631"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325623631"}]},"ts":"1676325623631"} 2023-02-13 22:00:23,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-13 22:00:23,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 in 191 msec 2023-02-13 22:00:23,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-13 22:00:23,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, ASSIGN in 350 msec 2023-02-13 22:00:23,639 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:00:23,640 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325623640"}]},"ts":"1676325623640"} 2023-02-13 22:00:23,641 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-13 22:00:23,643 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:00:23,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 404 msec 2023-02-13 22:00:23,746 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 22:00:23,751 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:49504, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 22:00:23,761 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:23,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 22:00:23,763 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 22:00:23,763 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:23,764 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 22:00:23,764 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 22:00:23,765 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,765 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,766 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:23,766 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,766 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:23,766 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:23,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:23,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:23,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:23,766 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,766 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,767 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:23,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,768 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:23,768 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:23,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:23,768 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:23,768 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:23,768 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:23,768 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:23,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,769 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:23,769 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:23,769 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:23,768 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:23,769 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:23,769 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:23,769 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:23,770 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:23,770 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:23,770 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:23,770 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:23,770 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:23,770 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. started... 2023-02-13 22:00:23,770 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:23,771 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:23,771 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fff278c72d1992f129cd33a6d722426 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 22:00:23,771 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:23,773 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,774 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,774 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:23,774 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,775 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,775 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:23,784 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/6df2b1bcc51c42648f672154ca70453d 2023-02-13 22:00:23,799 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/67c137e6cef04d86ae0911157e17a267 2023-02-13 22:00:23,806 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/6df2b1bcc51c42648f672154ca70453d as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d 2023-02-13 22:00:23,812 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:00:23,813 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/67c137e6cef04d86ae0911157e17a267 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267 2023-02-13 22:00:23,819 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:00:23,820 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4fff278c72d1992f129cd33a6d722426 in 49ms, sequenceid=6, compaction requested=false 2023-02-13 22:00:23,820 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-13 22:00:23,821 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fff278c72d1992f129cd33a6d722426: 2023-02-13 22:00:23,821 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:23,821 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:23,821 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:23,821 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,821 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:23,821 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:23,822 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,822 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:23,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:23,823 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,823 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:23,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:23,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:23,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:23,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:23,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:23,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:23,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 22:00:23,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66ff4396[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:23,825 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 22:00:23,825 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,825 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,825 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,825 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,826 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:23,826 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:23,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,826 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:23,826 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:23,826 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:23,826 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,826 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:23,826 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:23,826 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:23,826 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:23,826 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:23,827 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:23,827 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:23,827 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:23,827 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:23,827 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:23,828 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:23,828 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,828 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:23,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:23,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:23,828 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:23,828 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:23,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:23,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:23,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:23,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:23,832 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 22:00:23,832 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 22:00:23,832 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:23,832 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:23,832 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 22:00:23,832 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:23,837 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,837 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,837 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,837 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,837 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:23,837 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:23,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:23,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,837 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:23,838 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:23,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:23,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:23,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:23,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:23,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:23,838 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:23,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:23,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:23,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:23,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,839 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:23,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:23,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:23,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:23,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:23,841 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:23,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,841 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:23,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:23,841 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,848 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:23,848 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:23,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:23,848 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,848 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:23,848 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:23,848 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:23,848 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:23,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:23,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:23,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:23,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:23,849 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:23,848 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 22:00:23,850 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:23,848 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:23,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:23,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:23,849 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:23,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:23,850 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:23,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:23,850 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:23,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:23,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:23,850 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,850 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:23,851 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,851 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,851 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:23,851 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:23,851 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,851 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,951 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:23,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:23,979 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:23,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:23,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 22:00:23,981 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 22:00:23,981 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:23,982 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 22:00:23,982 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 22:00:23,982 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,982 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,983 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,983 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:23,983 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:23,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:23,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:23,983 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:23,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:23,983 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:23,983 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:23,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:23,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:23,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:23,985 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:23,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:23,985 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:23,985 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:23,985 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:23,985 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:23,985 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:23,985 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:23,986 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:23,985 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:23,986 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:23,986 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:23,986 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:23,986 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:23,986 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:23,986 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:23,987 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:23,986 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:23,987 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:23,987 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:23,987 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:23,987 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:23,987 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:23,987 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:23,987 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. started... 2023-02-13 22:00:23,988 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fff278c72d1992f129cd33a6d722426 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:23,995 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,998 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,998 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,998 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:23,998 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:23,998 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,001 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2 2023-02-13 22:00:24,016 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/8df92951b18149bd896d1cf26de7a274 2023-02-13 22:00:24,023 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2 2023-02-13 22:00:24,028 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:00:24,029 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/8df92951b18149bd896d1cf26de7a274 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274 2023-02-13 22:00:24,035 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:00:24,036 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fff278c72d1992f129cd33a6d722426 in 48ms, sequenceid=19, compaction requested=false 2023-02-13 22:00:24,036 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fff278c72d1992f129cd33a6d722426: 2023-02-13 22:00:24,036 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:24,036 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:24,036 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:24,036 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,036 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,036 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,038 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,038 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,038 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,038 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 22:00:24,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78dc2d26[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:24,040 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 22:00:24,040 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,041 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,041 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,041 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,041 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,041 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,041 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,041 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:24,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,041 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:24,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,041 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,041 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,041 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,041 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,041 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:24,041 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:24,043 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,043 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,043 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,044 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,044 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,044 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,044 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,045 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,044 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,045 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:24,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 22:00:24,049 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 22:00:24,049 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:24,049 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:24,049 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 22:00:24,049 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:24,054 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,054 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,054 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,054 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,055 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,054 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,055 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,055 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,055 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,056 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,058 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,058 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,058 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,060 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,060 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,061 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,061 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,060 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,060 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,061 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,061 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,061 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,062 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,062 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,062 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,061 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 22:00:24,062 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:24,061 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,063 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:24,063 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:24,062 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,063 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,063 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,063 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,063 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,163 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:24,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:24,187 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:24,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:24,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 22:00:24,189 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 22:00:24,189 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,190 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 22:00:24,190 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 22:00:24,190 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,190 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,191 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,191 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,191 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,191 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,192 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:24,191 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,193 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,193 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,193 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,193 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,193 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,193 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,193 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,193 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,193 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,193 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,195 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,195 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,195 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,196 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,195 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,196 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,196 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,196 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,197 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,197 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,197 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,196 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:24,197 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:24,197 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:24,197 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. started... 2023-02-13 22:00:24,197 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fff278c72d1992f129cd33a6d722426 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:24,204 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,206 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,206 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,206 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,207 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,207 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,213 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/6c1d22cb46204172b92357cfe3e3a653 2023-02-13 22:00:24,227 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/ba89463d751c446aa82324d582a2e5f6 2023-02-13 22:00:24,232 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/6c1d22cb46204172b92357cfe3e3a653 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653 2023-02-13 22:00:24,238 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:00:24,238 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/ba89463d751c446aa82324d582a2e5f6 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6 2023-02-13 22:00:24,244 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:00:24,245 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fff278c72d1992f129cd33a6d722426 in 48ms, sequenceid=32, compaction requested=true 2023-02-13 22:00:24,245 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fff278c72d1992f129cd33a6d722426: 2023-02-13 22:00:24,245 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:24,245 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:24,245 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:24,245 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,245 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,245 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,246 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,246 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,247 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,247 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 22:00:24,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f3ee411[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:24,249 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 22:00:24,249 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,250 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,250 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,250 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,251 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,251 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:24,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,253 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,253 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,253 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,254 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,254 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,254 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,254 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:24,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 22:00:24,259 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 22:00:24,259 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:24,259 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:24,259 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 22:00:24,259 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:24,264 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,264 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,264 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,264 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,265 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,264 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,264 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,265 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,265 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,266 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,268 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,269 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,269 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,273 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,273 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,273 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,273 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,273 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,273 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,273 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 22:00:24,274 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:24,273 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,273 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,273 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,275 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,275 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,275 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,275 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,275 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:24,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,275 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:24,275 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,275 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,276 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,276 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,276 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,375 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:24,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:24,403 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:24,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:24,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 22:00:24,405 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 22:00:24,405 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,405 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 22:00:24,405 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 22:00:24,406 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,406 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,407 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,408 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,408 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,407 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,408 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,408 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,408 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,408 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,408 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,409 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,409 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,409 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,409 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,409 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,411 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,411 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,411 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:24,412 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:24,412 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,412 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,412 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:24,412 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,412 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. started... 2023-02-13 22:00:24,413 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fff278c72d1992f129cd33a6d722426 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:24,420 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,423 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,423 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,423 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,424 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,424 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,427 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/e4a2002f589744f7b619efdfa1431f7f 2023-02-13 22:00:24,441 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/8d52b25243d44b63a4034e0caa36ff1c 2023-02-13 22:00:24,446 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/e4a2002f589744f7b619efdfa1431f7f as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f 2023-02-13 22:00:24,452 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:00:24,453 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/8d52b25243d44b63a4034e0caa36ff1c as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c 2023-02-13 22:00:24,457 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:00:24,458 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fff278c72d1992f129cd33a6d722426 in 46ms, sequenceid=45, compaction requested=true 2023-02-13 22:00:24,458 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fff278c72d1992f129cd33a6d722426: 2023-02-13 22:00:24,458 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:24,458 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:24,458 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:24,458 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,458 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,458 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,459 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,459 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,460 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,460 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 22:00:24,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79828c91[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:24,461 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 22:00:24,462 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,462 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,462 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,462 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,462 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,462 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:24,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,463 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,464 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,464 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,464 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,464 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,465 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,465 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,465 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,465 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:24,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 22:00:24,470 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 22:00:24,470 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:24,470 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:24,470 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 22:00:24,470 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:24,473 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,473 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,473 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,473 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,473 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,473 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,473 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,473 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,474 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,474 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,476 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,476 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,476 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,495 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,496 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 22:00:24,496 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:24,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,496 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,497 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:24,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,497 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:24,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,498 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,498 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,498 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,498 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,498 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,498 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,555 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 22:00:24,556 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 22:00:24,556 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:24,556 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-13 22:00:24,597 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:24,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:24,624 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:24,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:24,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 22:00:24,626 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 22:00:24,626 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,626 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 22:00:24,626 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 22:00:24,626 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,626 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,627 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,627 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,627 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,627 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,628 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,628 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:24,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,628 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,629 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,629 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,629 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,629 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,629 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,634 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,634 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,634 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:24,634 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,634 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,634 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. started... 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,634 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fff278c72d1992f129cd33a6d722426 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:24,634 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,640 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,640 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,640 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,640 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,640 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,640 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,647 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/535fb2c0715344c48bb6e9a46ac2e974 2023-02-13 22:00:24,660 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2 2023-02-13 22:00:24,666 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/535fb2c0715344c48bb6e9a46ac2e974 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974 2023-02-13 22:00:24,671 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:00:24,672 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2 2023-02-13 22:00:24,677 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:00:24,678 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fff278c72d1992f129cd33a6d722426 in 44ms, sequenceid=58, compaction requested=true 2023-02-13 22:00:24,679 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fff278c72d1992f129cd33a6d722426: 2023-02-13 22:00:24,679 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,680 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,680 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,681 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,681 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 22:00:24,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@657d93c4[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:24,683 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 22:00:24,684 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,684 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,684 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,684 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,685 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,685 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,685 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,685 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:24,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,685 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,685 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,685 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,685 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,685 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,685 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:24,685 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:24,686 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,686 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,686 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,686 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,687 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,687 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,687 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,687 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,687 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,687 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:24,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 22:00:24,690 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 22:00:24,690 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:24,690 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:24,690 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 22:00:24,690 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:24,692 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,692 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,692 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,692 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,692 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,692 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,693 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,692 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,693 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,694 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,695 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,696 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,696 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,707 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,707 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,707 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,707 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,707 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,707 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,707 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,707 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,707 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 22:00:24,707 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:24,707 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,707 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,708 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:24,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,708 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:24,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,809 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:24,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:24,834 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:24,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:24,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 22:00:24,836 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 22:00:24,836 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,836 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 22:00:24,836 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 22:00:24,836 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,836 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,837 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,837 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,837 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,837 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,838 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,838 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:24,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,838 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,838 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 22:00:24,839 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,839 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,839 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,839 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,839 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 22:00:24,839 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,839 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:24,840 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,840 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. started... 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,840 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,840 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fff278c72d1992f129cd33a6d722426 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:24,841 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,841 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,841 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,842 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,842 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,842 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,853 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/9f466043cc9a4f8f99850c687ad357ec 2023-02-13 22:00:24,870 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/f61d1f0327e24ac980309e4171aca6ec 2023-02-13 22:00:24,876 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/9f466043cc9a4f8f99850c687ad357ec as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec 2023-02-13 22:00:24,882 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:00:24,883 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/f61d1f0327e24ac980309e4171aca6ec as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec 2023-02-13 22:00:24,889 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:00:24,890 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fff278c72d1992f129cd33a6d722426 in 50ms, sequenceid=71, compaction requested=true 2023-02-13 22:00:24,890 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fff278c72d1992f129cd33a6d722426: 2023-02-13 22:00:24,890 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:24,890 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:24,890 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:24,890 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,890 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 22:00:24,890 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 22:00:24,892 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,892 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,892 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,892 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 22:00:24,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c513750[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:24,895 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 22:00:24,895 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,896 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,896 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,896 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,896 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,896 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:24,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,896 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,896 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,896 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,896 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:24,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,896 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,897 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 22:00:24,897 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:24,897 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 22:00:24,897 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:24,898 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,898 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,898 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,898 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,898 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,898 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,898 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 22:00:24,898 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,898 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,898 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 22:00:24,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:24,901 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 22:00:24,901 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 22:00:24,901 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:24,901 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:24,901 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 22:00:24,901 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:24,902 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,902 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,902 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,902 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:24,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,902 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,902 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:24,902 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,903 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,903 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:24,904 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:24,905 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 22:00:24,905 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,906 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,907 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,907 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:24,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,908 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:24,908 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 22:00:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 22:00:24,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:24,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:24,908 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:24,908 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:24,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:24,909 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:24,909 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:24,909 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 22:00:24,909 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 22:00:24,909 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:24,909 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:25,009 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:25,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:25,278 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 22:00:25,280 DEBUG [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:25,282 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,284 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,284 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,284 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,286 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:25,286 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:25,287 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:25,533 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,536 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:25,536 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:25,537 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:25,542 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-13 22:00:25,542 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:25,542 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:25,542 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:25,543 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:25,543 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:00:25,549 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 22:00:25,550 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:25,551 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:25,551 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:25,551 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:25,552 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:25,552 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:25,553 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:25,553 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:25,554 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:25,554 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,554 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:25,554 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,554 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,554 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,555 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-13 22:00:25,556 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:25,556 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:25,557 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325655557 2023-02-13 22:00:25,558 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,558 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:25,562 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:25,562 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=4fff278c72d1992f129cd33a6d722426, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:25,563 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:25,563 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325685563 2023-02-13 22:00:25,563 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325745563 2023-02-13 22:00:25,563 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 22:00:25,563 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:25,565 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:25,565 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:25,565 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:25,813 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,813 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:25,813 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:25,814 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:25,814 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:25,814 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:25,814 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:25,815 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:25,815 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:25,816 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:25,819 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 22:00:25,820 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:25,821 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:25,821 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:25,821 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:25,822 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:25,822 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:25,823 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:25,823 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:25,823 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:25,824 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,825 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:25,825 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,825 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,825 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,826 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 22:00:25,826 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:25,827 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:25,827 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325655827 2023-02-13 22:00:25,828 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:25,828 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,830 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:25,831 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=4fff278c72d1992f129cd33a6d722426, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:25,831 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:25,831 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325685831 2023-02-13 22:00:25,831 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325745831 2023-02-13 22:00:25,831 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:25,831 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:25,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN}] 2023-02-13 22:00:25,834 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:25,834 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:25,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:25,834 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:26,081 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,082 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:26,082 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:26,082 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:26,082 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:26,082 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:26,083 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:26,083 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:26,083 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:26,084 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:26,087 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 22:00:26,087 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:26,088 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:26,088 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:26,088 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:26,089 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,089 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:26,090 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,090 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,090 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:26,091 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,091 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:26,091 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,091 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,091 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,091 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 22:00:26,092 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:26,093 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:26,093 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325656093 2023-02-13 22:00:26,095 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,095 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:26,098 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:26,098 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=4fff278c72d1992f129cd33a6d722426, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:26,098 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:26,098 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325686098 2023-02-13 22:00:26,099 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325746099 2023-02-13 22:00:26,099 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:26,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN}] 2023-02-13 22:00:26,099 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:26,101 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,102 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4fff278c72d1992f129cd33a6d722426, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:26,102 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325626102"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325626102"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325626102"}]},"ts":"1676325626102"} 2023-02-13 22:00:26,104 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:26,104 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:26,104 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,104 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:26,104 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:26,349 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,349 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:26,349 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:26,350 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:26,350 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:26,351 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:26,351 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:26,351 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:26,351 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:26,353 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:26,359 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 22:00:26,359 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 22:00:26,360 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:26,361 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:26,362 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:26,362 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:26,363 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,363 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:26,363 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,363 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,364 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:26,364 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,364 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,365 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:26,365 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,365 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,365 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,365 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:26,365 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,365 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-13 22:00:26,366 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:26,366 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:26,366 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426 to restore RIT 2023-02-13 22:00:26,367 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325656367 2023-02-13 22:00:26,367 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:26,367 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,370 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:26,370 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=4fff278c72d1992f129cd33a6d722426, regionState=CLOSING, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:26,370 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:26,370 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325686370 2023-02-13 22:00:26,371 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325746371 2023-02-13 22:00:26,371 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:26,371 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4fff278c72d1992f129cd33a6d722426, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:26,371 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325626371"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325626371"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325626371"}]},"ts":"1676325626371"} 2023-02-13 22:00:26,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:26,373 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:26,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:26,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 4fff278c72d1992f129cd33a6d722426, disabling compactions & flushes 2023-02-13 22:00:26,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:26,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:26,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. after waiting 0 ms 2023-02-13 22:00:26,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:26,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 4fff278c72d1992f129cd33a6d722426 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 22:00:26,546 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/6c5a0256096b48df97aac6e11c9bb6c8 2023-02-13 22:00:26,564 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/47594576d14c4a86a515de11d88cd3b1 2023-02-13 22:00:26,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf1/6c5a0256096b48df97aac6e11c9bb6c8 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8 2023-02-13 22:00:26,574 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:00:26,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/.tmp/cf2/47594576d14c4a86a515de11d88cd3b1 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1 2023-02-13 22:00:26,581 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:00:26,583 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4fff278c72d1992f129cd33a6d722426 in 54ms, sequenceid=82, compaction requested=true 2023-02-13 22:00:26,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 22:00:26,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:26,603 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. 2023-02-13 22:00:26,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 4fff278c72d1992f129cd33a6d722426: Waiting for close lock at 1676325626527Running coprocessor pre-close hooks at 1676325626527Disabling compacts and flushes for region at 1676325626527Disabling writes for close at 1676325626527Obtaining lock to block concurrent updates at 1676325626528 (+1 ms)Preparing flush snapshotting stores in 4fff278c72d1992f129cd33a6d722426 at 1676325626528Finished memstore snapshotting testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676325626528Flushing stores of testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. at 1676325626529 (+1 ms)Flushing 4fff278c72d1992f129cd33a6d722426/cf1: creating writer at 1676325626531 (+2 ms)Flushing 4fff278c72d1992f129cd33a6d722426/cf1: appending metadata at 1676325626536 (+5 ms)Flushing 4fff278c72d1992f129cd33a6d722426/cf1: closing flushed file at 1676325626537 (+1 ms)Flushing 4fff278c72d1992f129cd33a6d722426/cf2: creating writer at 1676325626552 (+15 ms)Flushing 4fff278c72d1992f129cd33a6d722426/cf2: appending metadata at 1676325626554 (+2 ms)Flushing 4fff278c72d1992f129cd33a6d722426/cf2: closing flushed file at 1676325626554Flushing 4fff278c72d1992f129cd33a6d722426/cf1: reopening flushed file at 1676325626570 (+16 ms)Flushing 4fff278c72d1992f129cd33a6d722426/cf2: reopening flushed file at 1676325626576 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4fff278c72d1992f129cd33a6d722426 in 54ms, sequenceid=82, compaction requested=true at 1676325626583 (+7 ms)Writing region close event to WAL at 1676325626599 (+16 ms)Running coprocessor post-close hooks at 1676325626602 (+3 ms)Closed at 1676325626603 (+1 ms) 2023-02-13 22:00:26,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:26,606 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4fff278c72d1992f129cd33a6d722426, regionState=CLOSED 2023-02-13 22:00:26,606 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325626606"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325626606"}]},"ts":"1676325626606"} 2023-02-13 22:00:26,608 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:26,608 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:26,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:26,608 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:26,621 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,621 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:26,621 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:26,621 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:26,621 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:26,621 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:26,622 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:26,622 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:26,622 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:26,622 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:00:26,624 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 22:00:26,624 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 22:00:26,624 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 22:00:26,624 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:26,625 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:26,625 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:26,625 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:26,625 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,626 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:26,626 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,626 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,626 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:26,627 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,627 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,627 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:26,627 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:26,628 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,628 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,628 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,628 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:26,628 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,628 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:26,628 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 22:00:26,629 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:26,629 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:26,629 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426 to restore RIT 2023-02-13 22:00:26,630 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325656630 2023-02-13 22:00:26,630 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:26,630 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:26,633 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:26,633 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=4fff278c72d1992f129cd33a6d722426, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:26,633 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:26,633 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325686633 2023-02-13 22:00:26,633 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325746633 2023-02-13 22:00:26,633 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:26,633 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4fff278c72d1992f129cd33a6d722426, regionState=CLOSED, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:26,633 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325626633"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325626633"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325626633"}]},"ts":"1676325626633"} 2023-02-13 22:00:26,635 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:26,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-13 22:00:26,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 in 262 msec 2023-02-13 22:00:26,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:26,638 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:26,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:26,883 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,884 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:26,884 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:26,884 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:26,885 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:26,886 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:26,886 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:26,886 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:26,886 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:26,888 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 2 msec 2023-02-13 22:00:26,895 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 22:00:26,895 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 22:00:26,895 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:26,895 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:26,896 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:26,896 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:26,896 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:26,897 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,897 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:26,897 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,898 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:26,898 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:26,899 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,899 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,899 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:26,899 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:26,899 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,899 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,899 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,899 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:26,900 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:26,900 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:00:26,900 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:26,901 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:26,901 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426 to restore RIT 2023-02-13 22:00:26,902 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325656901 2023-02-13 22:00:26,902 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,902 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:26,905 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:26,905 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=4fff278c72d1992f129cd33a6d722426, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:26,905 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:26,905 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325686905 2023-02-13 22:00:26,905 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325746905 2023-02-13 22:00:26,905 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:26,906 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:26,907 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:26,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-13 22:00:26,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN in 808 msec 2023-02-13 22:00:26,913 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:26,913 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:26,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:26,913 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:27,156 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,156 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:27,156 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:27,156 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:27,157 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:27,157 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:27,158 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:27,158 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:27,158 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:27,160 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:27,163 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 22:00:27,163 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:27,164 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:27,164 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:27,165 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:27,166 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:27,166 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:27,167 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:27,168 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:27,169 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:27,169 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:27,170 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:27,171 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,171 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:27,171 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:27,171 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:27,171 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:27,171 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:27,171 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-13 22:00:27,172 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:27,173 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:27,173 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325657173 2023-02-13 22:00:27,173 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:27,174 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,177 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:27,177 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=4fff278c72d1992f129cd33a6d722426, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:27,177 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:27,177 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325687177 2023-02-13 22:00:27,177 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325747177 2023-02-13 22:00:27,177 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 22:00:27,177 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,177 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,177 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,183 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:27,192 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=4fff278c72d1992f129cd33a6d722426, threads=14 2023-02-13 22:00:27,193 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,193 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,195 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,195 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,196 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,196 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,196 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,196 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,196 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,197 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,197 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,199 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,200 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,200 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,306 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,306 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,307 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,331 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,339 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,343 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,349 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,355 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,355 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,357 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,357 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,358 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,360 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,530 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 22:00:27,723 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,724 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 4fff278c72d1992f129cd33a6d722426 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 22:00:27,773 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:27,773 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,773 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:27,773 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:27,927 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,928 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:27,928 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:27,928 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:27,928 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:27,929 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:27,929 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:27,929 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:27,929 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:27,930 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:27,932 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 22:00:27,933 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:27,933 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:27,934 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:27,934 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:27,935 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:27,935 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:27,936 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:27,937 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:27,937 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:27,938 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:27,938 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:27,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:27,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:27,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:27,939 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,939 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,939 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:27,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:27,940 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 22:00:27,940 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:27,941 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:27,941 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325657941 2023-02-13 22:00:27,942 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:27,942 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:27,944 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:27,945 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=4fff278c72d1992f129cd33a6d722426, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:27,945 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:27,945 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325687945 2023-02-13 22:00:27,945 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325747945 2023-02-13 22:00:27,945 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:27,958 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=4fff278c72d1992f129cd33a6d722426, threads=14 2023-02-13 22:00:27,959 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,959 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,959 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,959 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,959 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,959 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,960 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,963 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,963 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,963 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,963 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,963 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,963 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:27,964 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,089 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,103 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,114 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,131 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,132 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,138 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,152 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,153 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,155 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,155 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,160 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,160 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,163 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,165 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6 for region: 4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:28,165 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 4fff278c72d1992f129cd33a6d722426 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 22:00:28,204 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:28,210 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:00:28,212 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:00:28,212 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:28,212 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:28,212 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:28,445 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,446 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:28,446 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:28,446 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:28,447 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:28,447 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:28,447 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:28,447 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:28,448 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:28,450 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:28,453 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 22:00:28,453 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:28,454 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:28,454 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:28,455 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:28,455 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:28,455 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:28,456 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:28,456 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:28,456 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:28,456 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:28,457 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:28,457 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,457 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:28,457 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:28,457 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:28,458 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,458 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,458 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,458 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:28,458 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:28,458 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 22:00:28,459 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:28,459 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:28,459 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325658459 2023-02-13 22:00:28,461 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:28,461 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,463 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:28,463 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=4fff278c72d1992f129cd33a6d722426, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:28,463 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:28,463 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325688463 2023-02-13 22:00:28,464 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325748464 2023-02-13 22:00:28,464 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:28,467 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:28,469 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:28,469 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:28,469 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:28,714 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:28,714 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:28,714 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:28,714 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:28,715 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:28,715 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:28,715 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:28,715 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:28,715 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:28,717 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:28,719 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 22:00:28,719 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:28,719 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:28,720 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:28,721 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:28,721 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:28,721 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:28,722 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:28,723 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:28,723 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:28,724 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:28,725 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:28,726 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,727 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN 2023-02-13 22:00:28,727 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:28,727 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:28,727 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,727 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,727 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,727 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fff278c72d1992f129cd33a6d722426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:28,727 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4fff278c72d1992f129cd33a6d722426, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:28,727 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:00:28,728 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:28,728 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:28,729 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325658729 2023-02-13 22:00:28,729 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:28,730 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:28,732 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:28,732 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=4fff278c72d1992f129cd33a6d722426, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:28,732 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:28,732 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325688732 2023-02-13 22:00:28,732 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325748732 2023-02-13 22:00:28,732 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:28,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325628734"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676325628734"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676325628734"}]},"ts":"1676325628734"} 2023-02-13 22:00:28,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325628734"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325628734"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325628734"}]},"ts":"1676325628734"} 2023-02-13 22:00:28,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325628734"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325628734"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325628734"}]},"ts":"1676325628734"} 2023-02-13 22:00:28,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b458c667c868fe528fcdd3363340873b, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=245a056601be8d7d102a0dc6a730bbad, ASSIGN}] 2023-02-13 22:00:28,742 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=245a056601be8d7d102a0dc6a730bbad, ASSIGN 2023-02-13 22:00:28,743 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=245a056601be8d7d102a0dc6a730bbad, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:00:28,744 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b458c667c868fe528fcdd3363340873b, ASSIGN 2023-02-13 22:00:28,745 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b458c667c868fe528fcdd3363340873b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:00:28,894 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 22:00:28,896 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=245a056601be8d7d102a0dc6a730bbad, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:28,896 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325628896"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325628896"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325628896"}]},"ts":"1676325628896"} 2023-02-13 22:00:28,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 245a056601be8d7d102a0dc6a730bbad, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:28,907 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=b458c667c868fe528fcdd3363340873b, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:28,907 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325628907"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325628907"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325628907"}]},"ts":"1676325628907"} 2023-02-13 22:00:28,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure b458c667c868fe528fcdd3363340873b, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:29,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 245a056601be8d7d102a0dc6a730bbad, NAME => 'testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 22:00:29,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:29,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:29,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,063 INFO [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,065 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1 2023-02-13 22:00:29,065 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1 2023-02-13 22:00:29,066 INFO [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 245a056601be8d7d102a0dc6a730bbad columnFamilyName cf1 2023-02-13 22:00:29,085 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974-top 2023-02-13 22:00:29,090 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653-top 2023-02-13 22:00:29,094 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8-top 2023-02-13 22:00:29,099 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d-top 2023-02-13 22:00:29,125 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec-top 2023-02-13 22:00:29,131 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2-top 2023-02-13 22:00:29,136 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f-top 2023-02-13 22:00:29,136 INFO [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(310): Store=245a056601be8d7d102a0dc6a730bbad/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:29,136 INFO [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,137 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2 2023-02-13 22:00:29,137 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2 2023-02-13 22:00:29,137 INFO [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 245a056601be8d7d102a0dc6a730bbad columnFamilyName cf2 2023-02-13 22:00:29,154 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1-top 2023-02-13 22:00:29,159 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2-top 2023-02-13 22:00:29,166 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267-top 2023-02-13 22:00:29,173 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c-top 2023-02-13 22:00:29,179 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274-top 2023-02-13 22:00:29,184 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6-top 2023-02-13 22:00:29,189 DEBUG [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec-top 2023-02-13 22:00:29,189 INFO [StoreOpener-245a056601be8d7d102a0dc6a730bbad-1] regionserver.HStore(310): Store=245a056601be8d7d102a0dc6a730bbad/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:29,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:29,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 245a056601be8d7d102a0dc6a730bbad; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66448379, jitterRate=-0.00984199345111847}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:29,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 245a056601be8d7d102a0dc6a730bbad: Running coprocessor pre-open hook at 1676325629061Writing region info on filesystem at 1676325629061Initializing all the Stores at 1676325629063 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325629063Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325629063Cleaning up temporary data from old regions at 1676325629192 (+129 ms)Cleaning up detritus from prior splits at 1676325629192Running coprocessor post-open hooks at 1676325629195 (+3 ms)Region opened successfully at 1676325629195 2023-02-13 22:00:29,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad., pid=33, masterSystemTime=1676325629055 2023-02-13 22:00:29,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:29,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:29,198 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:29,198 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:29,198 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,198 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,198 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1912): 245a056601be8d7d102a0dc6a730bbad/cf2 is initiating minor compaction (all files) 2023-02-13 22:00:29,198 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1912): 245a056601be8d7d102a0dc6a730bbad/cf1 is initiating minor compaction (all files) 2023-02-13 22:00:29,198 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 245a056601be8d7d102a0dc6a730bbad/cf2 in testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,198 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 245a056601be8d7d102a0dc6a730bbad/cf1 in testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,198 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8-top] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/.tmp, totalSize=34.9 K 2023-02-13 22:00:29,198 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1-top] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/.tmp, totalSize=34.9 K 2023-02-13 22:00:29,199 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676325623751 2023-02-13 22:00:29,199 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676325623751 2023-02-13 22:00:29,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,200 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676325623956 2023-02-13 22:00:29,200 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676325623956 2023-02-13 22:00:29,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b458c667c868fe528fcdd3363340873b, NAME => 'testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 22:00:29,200 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=245a056601be8d7d102a0dc6a730bbad, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:29,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:29,200 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325629200"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325629200"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325629200"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325629200"}]},"ts":"1676325629200"} 2023-02-13 22:00:29,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:29,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,202 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676325624165 2023-02-13 22:00:29,202 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676325624165 2023-02-13 22:00:29,202 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676325624379 2023-02-13 22:00:29,202 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676325624379 2023-02-13 22:00:29,203 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676325624601 2023-02-13 22:00:29,203 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676325624601 2023-02-13 22:00:29,204 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676325624813 2023-02-13 22:00:29,204 INFO [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,204 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676325624813 2023-02-13 22:00:29,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-13 22:00:29,205 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1 2023-02-13 22:00:29,205 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1 2023-02-13 22:00:29,205 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676325625011 2023-02-13 22:00:29,205 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676325625011 2023-02-13 22:00:29,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 245a056601be8d7d102a0dc6a730bbad, server=jenkins-hbase20.apache.org,33193,1676325615429 in 302 msec 2023-02-13 22:00:29,206 INFO [StoreOpener-b458c667c868fe528fcdd3363340873b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b458c667c868fe528fcdd3363340873b columnFamilyName cf1 2023-02-13 22:00:29,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=245a056601be8d7d102a0dc6a730bbad, ASSIGN in 467 msec 2023-02-13 22:00:29,224 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] throttle.PressureAwareThroughputController(145): 245a056601be8d7d102a0dc6a730bbad#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:29,230 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] throttle.PressureAwareThroughputController(145): 245a056601be8d7d102a0dc6a730bbad#cf1#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:29,235 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974-bottom 2023-02-13 22:00:29,240 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653-bottom 2023-02-13 22:00:29,245 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8-bottom 2023-02-13 22:00:29,251 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d-bottom 2023-02-13 22:00:29,260 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec-bottom 2023-02-13 22:00:29,265 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2-bottom 2023-02-13 22:00:29,266 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/.tmp/cf2/dbba82911616428391bb2e9f23a6967d as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/dbba82911616428391bb2e9f23a6967d 2023-02-13 22:00:29,267 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/.tmp/cf1/a7f9bf15a5af4a05a39888a99549f6d3 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/a7f9bf15a5af4a05a39888a99549f6d3 2023-02-13 22:00:29,270 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f-bottom 2023-02-13 22:00:29,271 INFO [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(310): Store=b458c667c868fe528fcdd3363340873b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:29,271 INFO [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,272 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2 2023-02-13 22:00:29,272 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2 2023-02-13 22:00:29,273 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 245a056601be8d7d102a0dc6a730bbad/cf2 of 245a056601be8d7d102a0dc6a730bbad into dbba82911616428391bb2e9f23a6967d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:29,273 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 245a056601be8d7d102a0dc6a730bbad: 2023-02-13 22:00:29,273 INFO [StoreOpener-b458c667c868fe528fcdd3363340873b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b458c667c868fe528fcdd3363340873b columnFamilyName cf2 2023-02-13 22:00:29,273 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad., storeName=245a056601be8d7d102a0dc6a730bbad/cf2, priority=9, startTime=1676325629197; duration=0sec 2023-02-13 22:00:29,273 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:29,274 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 245a056601be8d7d102a0dc6a730bbad/cf1 of 245a056601be8d7d102a0dc6a730bbad into a7f9bf15a5af4a05a39888a99549f6d3(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:29,274 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 245a056601be8d7d102a0dc6a730bbad: 2023-02-13 22:00:29,274 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad., storeName=245a056601be8d7d102a0dc6a730bbad/cf1, priority=9, startTime=1676325629196; duration=0sec 2023-02-13 22:00:29,274 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:29,289 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1-bottom 2023-02-13 22:00:29,293 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2-bottom 2023-02-13 22:00:29,296 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267-bottom 2023-02-13 22:00:29,300 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c-bottom 2023-02-13 22:00:29,304 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274-bottom 2023-02-13 22:00:29,309 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6-bottom 2023-02-13 22:00:29,313 DEBUG [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec-bottom 2023-02-13 22:00:29,313 INFO [StoreOpener-b458c667c868fe528fcdd3363340873b-1] regionserver.HStore(310): Store=b458c667c868fe528fcdd3363340873b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:29,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:29,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened b458c667c868fe528fcdd3363340873b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61265408, jitterRate=-0.08707427978515625}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:29,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for b458c667c868fe528fcdd3363340873b: Running coprocessor pre-open hook at 1676325629201Writing region info on filesystem at 1676325629201Initializing all the Stores at 1676325629202 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325629202Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325629204 (+2 ms)Cleaning up temporary data from old regions at 1676325629316 (+112 ms)Cleaning up detritus from prior splits at 1676325629316Running coprocessor post-open hooks at 1676325629319 (+3 ms)Region opened successfully at 1676325629319 2023-02-13 22:00:29,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b., pid=34, masterSystemTime=1676325629055 2023-02-13 22:00:29,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:29,320 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:29,320 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:29,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:29,320 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,320 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,321 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1912): b458c667c868fe528fcdd3363340873b/cf1 is initiating minor compaction (all files) 2023-02-13 22:00:29,320 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1912): b458c667c868fe528fcdd3363340873b/cf2 is initiating minor compaction (all files) 2023-02-13 22:00:29,321 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b458c667c868fe528fcdd3363340873b/cf1 in testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,321 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b458c667c868fe528fcdd3363340873b/cf2 in testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,321 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8-bottom] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/.tmp, totalSize=34.9 K 2023-02-13 22:00:29,321 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1-bottom] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/.tmp, totalSize=34.9 K 2023-02-13 22:00:29,322 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325623751 2023-02-13 22:00:29,322 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325623751 2023-02-13 22:00:29,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,322 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325623956 2023-02-13 22:00:29,322 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325623956 2023-02-13 22:00:29,323 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=b458c667c868fe528fcdd3363340873b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:29,323 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325624165 2023-02-13 22:00:29,323 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325624165 2023-02-13 22:00:29,323 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325629322"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325629322"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325629322"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325629322"}]},"ts":"1676325629322"} 2023-02-13 22:00:29,323 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325624379 2023-02-13 22:00:29,323 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325624379 2023-02-13 22:00:29,324 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325624601 2023-02-13 22:00:29,324 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325624601 2023-02-13 22:00:29,324 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325624813 2023-02-13 22:00:29,324 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325624813 2023-02-13 22:00:29,325 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325625011 2023-02-13 22:00:29,325 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325625011 2023-02-13 22:00:29,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-13 22:00:29,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure b458c667c868fe528fcdd3363340873b, server=jenkins-hbase20.apache.org,33193,1676325615429 in 415 msec 2023-02-13 22:00:29,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-13 22:00:29,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b458c667c868fe528fcdd3363340873b, ASSIGN in 589 msec 2023-02-13 22:00:29,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad in 4.0520 sec 2023-02-13 22:00:29,341 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] throttle.PressureAwareThroughputController(145): b458c667c868fe528fcdd3363340873b#cf2#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:29,342 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] throttle.PressureAwareThroughputController(145): b458c667c868fe528fcdd3363340873b#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:29,361 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/.tmp/cf2/3f824f8b43e94f359246f2e3e64eda94 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/3f824f8b43e94f359246f2e3e64eda94 2023-02-13 22:00:29,363 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/.tmp/cf1/d2971feaff1e4babb83942e2ecec903f as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/d2971feaff1e4babb83942e2ecec903f 2023-02-13 22:00:29,367 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b458c667c868fe528fcdd3363340873b/cf2 of b458c667c868fe528fcdd3363340873b into 3f824f8b43e94f359246f2e3e64eda94(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:29,367 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b458c667c868fe528fcdd3363340873b: 2023-02-13 22:00:29,367 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b., storeName=b458c667c868fe528fcdd3363340873b/cf2, priority=9, startTime=1676325629320; duration=0sec 2023-02-13 22:00:29,367 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:29,369 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b458c667c868fe528fcdd3363340873b/cf1 of b458c667c868fe528fcdd3363340873b into d2971feaff1e4babb83942e2ecec903f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:29,369 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b458c667c868fe528fcdd3363340873b: 2023-02-13 22:00:29,369 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b., storeName=b458c667c868fe528fcdd3363340873b/cf1, priority=9, startTime=1676325629319; duration=0sec 2023-02-13 22:00:29,369 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:29,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33193] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 148.251.75.209:49504 deadline: 1676325689483, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426. is not online on jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:29,611 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 22:00:29,618 INFO [Listener at localhost.localdomain/42295] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 22:00:29,620 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 22:00:29,639 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:29,641 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-13 22:00:29,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitWithoutPONR 2023-02-13 22:00:29,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:29,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 22:00:29,645 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325629645"}]},"ts":"1676325629645"} 2023-02-13 22:00:29,647 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-13 22:00:29,648 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-13 22:00:29,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b458c667c868fe528fcdd3363340873b, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=245a056601be8d7d102a0dc6a730bbad, UNASSIGN}] 2023-02-13 22:00:29,650 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=245a056601be8d7d102a0dc6a730bbad, UNASSIGN 2023-02-13 22:00:29,650 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=245a056601be8d7d102a0dc6a730bbad, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:29,650 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325629650"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325629650"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325629650"}]},"ts":"1676325629650"} 2023-02-13 22:00:29,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 245a056601be8d7d102a0dc6a730bbad, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:29,654 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b458c667c868fe528fcdd3363340873b, UNASSIGN 2023-02-13 22:00:29,654 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=b458c667c868fe528fcdd3363340873b, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:29,654 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325629654"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325629654"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325629654"}]},"ts":"1676325629654"} 2023-02-13 22:00:29,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure b458c667c868fe528fcdd3363340873b, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:29,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 22:00:29,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 245a056601be8d7d102a0dc6a730bbad, disabling compactions & flushes 2023-02-13 22:00:29,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. after waiting 0 ms 2023-02-13 22:00:29,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,818 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8-top] to archive 2023-02-13 22:00:29,819 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:29,821 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,823 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,824 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,825 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,826 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,827 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,829 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,830 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1-top] to archive 2023-02-13 22:00:29,831 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:29,832 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,833 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,834 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,835 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,836 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,837 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,838 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,842 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 22:00:29,842 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:29,842 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad. 2023-02-13 22:00:29,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 245a056601be8d7d102a0dc6a730bbad: Waiting for close lock at 1676325629804Running coprocessor pre-close hooks at 1676325629804Disabling compacts and flushes for region at 1676325629804Disabling writes for close at 1676325629805 (+1 ms)Writing region close event to WAL at 1676325629839 (+34 ms)Running coprocessor post-close hooks at 1676325629842 (+3 ms)Closed at 1676325629842 2023-02-13 22:00:29,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing b458c667c868fe528fcdd3363340873b, disabling compactions & flushes 2023-02-13 22:00:29,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. after waiting 0 ms 2023-02-13 22:00:29,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,844 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=245a056601be8d7d102a0dc6a730bbad, regionState=CLOSED 2023-02-13 22:00:29,845 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325629844"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325629844"}]},"ts":"1676325629844"} 2023-02-13 22:00:29,846 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8-bottom] to archive 2023-02-13 22:00:29,847 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:29,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-13 22:00:29,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 245a056601be8d7d102a0dc6a730bbad, server=jenkins-hbase20.apache.org,33193,1676325615429 in 194 msec 2023-02-13 22:00:29,849 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,850 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=245a056601be8d7d102a0dc6a730bbad, UNASSIGN in 199 msec 2023-02-13 22:00:29,851 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,852 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,853 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,854 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,855 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,856 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,858 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1-bottom] to archive 2023-02-13 22:00:29,858 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:29,860 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,861 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,862 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,863 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,864 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,865 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,866 DEBUG [StoreCloser-testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 22:00:29,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:29,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b. 2023-02-13 22:00:29,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for b458c667c868fe528fcdd3363340873b: Waiting for close lock at 1676325629844Running coprocessor pre-close hooks at 1676325629844Disabling compacts and flushes for region at 1676325629844Disabling writes for close at 1676325629844Writing region close event to WAL at 1676325629867 (+23 ms)Running coprocessor post-close hooks at 1676325629871 (+4 ms)Closed at 1676325629871 2023-02-13 22:00:29,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,874 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=b458c667c868fe528fcdd3363340873b, regionState=CLOSED 2023-02-13 22:00:29,874 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325629874"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325629874"}]},"ts":"1676325629874"} 2023-02-13 22:00:29,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-13 22:00:29,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure b458c667c868fe528fcdd3363340873b, server=jenkins-hbase20.apache.org,33193,1676325615429 in 220 msec 2023-02-13 22:00:29,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-13 22:00:29,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b458c667c868fe528fcdd3363340873b, UNASSIGN in 229 msec 2023-02-13 22:00:29,883 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325629883"}]},"ts":"1676325629883"} 2023-02-13 22:00:29,885 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-13 22:00:29,886 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-13 22:00:29,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 246 msec 2023-02-13 22:00:29,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 22:00:29,950 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-13 22:00:29,952 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitWithoutPONR 2023-02-13 22:00:29,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:29,958 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:29,960 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:29,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 22:00:29,968 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,968 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:29,968 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:29,971 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/recovered.edits] 2023-02-13 22:00:29,971 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/recovered.edits] 2023-02-13 22:00:29,971 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/recovered.edits] 2023-02-13 22:00:29,974 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/.regioninfo to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/.regioninfo 2023-02-13 22:00:29,974 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/.regioninfo to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/.regioninfo 2023-02-13 22:00:29,975 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/535fb2c0715344c48bb6e9a46ac2e974 2023-02-13 22:00:29,976 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c1d22cb46204172b92357cfe3e3a653 2023-02-13 22:00:29,977 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,977 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/535fb2c0715344c48bb6e9a46ac2e974.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,978 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6c5a0256096b48df97aac6e11c9bb6c8 2023-02-13 22:00:29,978 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,979 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/6c1d22cb46204172b92357cfe3e3a653.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,979 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/6df2b1bcc51c42648f672154ca70453d 2023-02-13 22:00:29,980 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,980 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/6c5a0256096b48df97aac6e11c9bb6c8.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,981 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/9f466043cc9a4f8f99850c687ad357ec 2023-02-13 22:00:29,981 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,982 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/6df2b1bcc51c42648f672154ca70453d.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,982 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2 2023-02-13 22:00:29,983 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,983 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/9f466043cc9a4f8f99850c687ad357ec.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,984 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf1/e4a2002f589744f7b619efdfa1431f7f 2023-02-13 22:00:29,984 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,985 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/e070ae2fe8a6487c91bf2629eb6ca1a2.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,986 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,986 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf1/e4a2002f589744f7b619efdfa1431f7f.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,988 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/47594576d14c4a86a515de11d88cd3b1 2023-02-13 22:00:29,990 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2 2023-02-13 22:00:29,991 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,991 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/47594576d14c4a86a515de11d88cd3b1.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,992 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/67c137e6cef04d86ae0911157e17a267 2023-02-13 22:00:29,992 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,992 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/49ffa7be2b0d4b9ba3625eb30184a8e2.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,993 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8d52b25243d44b63a4034e0caa36ff1c 2023-02-13 22:00:29,993 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,994 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/67c137e6cef04d86ae0911157e17a267.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,995 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,995 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/8df92951b18149bd896d1cf26de7a274 2023-02-13 22:00:29,995 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/8d52b25243d44b63a4034e0caa36ff1c.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,996 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,996 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/ba89463d751c446aa82324d582a2e5f6 2023-02-13 22:00:29,997 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/8df92951b18149bd896d1cf26de7a274.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,997 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,997 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/cf2/f61d1f0327e24ac980309e4171aca6ec 2023-02-13 22:00:29,998 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/ba89463d751c446aa82324d582a2e5f6.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,999 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/245a056601be8d7d102a0dc6a730bbad/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:29,999 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/b458c667c868fe528fcdd3363340873b/cf2/f61d1f0327e24ac980309e4171aca6ec.4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:30,001 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/recovered.edits/85.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426/recovered.edits/85.seqid 2023-02-13 22:00:30,001 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/a7f9bf15a5af4a05a39888a99549f6d3 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf1/a7f9bf15a5af4a05a39888a99549f6d3 2023-02-13 22:00:30,001 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/4fff278c72d1992f129cd33a6d722426 2023-02-13 22:00:30,002 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/d2971feaff1e4babb83942e2ecec903f to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf1/d2971feaff1e4babb83942e2ecec903f 2023-02-13 22:00:30,004 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/dbba82911616428391bb2e9f23a6967d to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/cf2/dbba82911616428391bb2e9f23a6967d 2023-02-13 22:00:30,005 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/3f824f8b43e94f359246f2e3e64eda94 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/cf2/3f824f8b43e94f359246f2e3e64eda94 2023-02-13 22:00:30,008 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/recovered.edits/90.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad/recovered.edits/90.seqid 2023-02-13 22:00:30,008 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/recovered.edits/90.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b/recovered.edits/90.seqid 2023-02-13 22:00:30,009 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:30,009 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitWithoutPONR/b458c667c868fe528fcdd3363340873b 2023-02-13 22:00:30,009 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 22:00:30,011 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:30,013 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-13 22:00:30,015 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-13 22:00:30,017 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:30,017 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-13 22:00:30,017 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325630017"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:30,017 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325630017"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:30,017 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325630017"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:30,019 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 22:00:30,019 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4fff278c72d1992f129cd33a6d722426, NAME => 'testSplitWithoutPONR,,1676325623239.4fff278c72d1992f129cd33a6d722426.', STARTKEY => '', ENDKEY => ''}, {ENCODED => b458c667c868fe528fcdd3363340873b, NAME => 'testSplitWithoutPONR,,1676325625280.b458c667c868fe528fcdd3363340873b.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 245a056601be8d7d102a0dc6a730bbad, NAME => 'testSplitWithoutPONR,41,1676325625280.245a056601be8d7d102a0dc6a730bbad.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 22:00:30,019 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-13 22:00:30,019 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676325630019"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:30,021 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-13 22:00:30,023 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:30,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 71 msec 2023-02-13 22:00:30,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 22:00:30,063 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-13 22:00:30,076 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=400 (was 388) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:34182 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:39580 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase20:33193-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1357b8ff-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:54344 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1357b8ff-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:39628 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:54418 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:34176 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:34166 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:54360 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:34184 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:55294 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:54402 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:55282 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:54370 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:39600 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:39606 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1490963902_15 at /127.0.0.1:39608 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=846 (was 835) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 282), ProcessCount=164 (was 164), AvailableMemoryMB=7419 (was 7556) 2023-02-13 22:00:30,090 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=400, OpenFileDescriptor=846, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=164, AvailableMemoryMB=7418 2023-02-13 22:00:30,091 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:30,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:00:30,093 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-13 22:00:30,095 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:30,097 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:00:30,097 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:00:30,099 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:30,099 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1 empty. 2023-02-13 22:00:30,100 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:30,100 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 22:00:30,110 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:30,112 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 386c949cc4d06cf801b328070dc3f9d1, NAME => 'testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:00:30,120 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:30,120 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 386c949cc4d06cf801b328070dc3f9d1, disabling compactions & flushes 2023-02-13 22:00:30,120 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,120 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,120 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. after waiting 0 ms 2023-02-13 22:00:30,120 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,120 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,120 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 386c949cc4d06cf801b328070dc3f9d1: Waiting for close lock at 1676325630120Disabling compacts and flushes for region at 1676325630120Disabling writes for close at 1676325630120Writing region close event to WAL at 1676325630120Closed at 1676325630120 2023-02-13 22:00:30,122 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:00:30,123 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325630123"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325630123"}]},"ts":"1676325630123"} 2023-02-13 22:00:30,124 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:00:30,125 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:00:30,125 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325630125"}]},"ts":"1676325630125"} 2023-02-13 22:00:30,127 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 22:00:30,130 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:30,131 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:30,131 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:30,131 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:30,131 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:30,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN}] 2023-02-13 22:00:30,133 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:30,134 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:00:30,284 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:30,286 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,287 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325630286"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325630286"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325630286"}]},"ts":"1676325630286"} 2023-02-13 22:00:30,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:30,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 386c949cc4d06cf801b328070dc3f9d1, NAME => 'testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:30,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:30,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:30,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:30,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:30,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:30,457 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:30,458 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1 2023-02-13 22:00:30,458 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1 2023-02-13 22:00:30,459 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 386c949cc4d06cf801b328070dc3f9d1 columnFamilyName cf1 2023-02-13 22:00:30,459 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(310): Store=386c949cc4d06cf801b328070dc3f9d1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:30,459 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:30,461 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2 2023-02-13 22:00:30,461 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2 2023-02-13 22:00:30,461 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 386c949cc4d06cf801b328070dc3f9d1 columnFamilyName cf2 2023-02-13 22:00:30,461 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(310): Store=386c949cc4d06cf801b328070dc3f9d1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:30,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:30,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:30,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:30,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:30,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:30,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 386c949cc4d06cf801b328070dc3f9d1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71895610, jitterRate=0.0713280737400055}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:30,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 386c949cc4d06cf801b328070dc3f9d1: Running coprocessor pre-open hook at 1676325630454Writing region info on filesystem at 1676325630454Initializing all the Stores at 1676325630456 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325630456Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325630456Cleaning up temporary data from old regions at 1676325630463 (+7 ms)Cleaning up detritus from prior splits at 1676325630464 (+1 ms)Running coprocessor post-open hooks at 1676325630468 (+4 ms)Region opened successfully at 1676325630468 2023-02-13 22:00:30,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1., pid=43, masterSystemTime=1676325630442 2023-02-13 22:00:30,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,470 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,470 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325630470"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325630470"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325630470"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325630470"}]},"ts":"1676325630470"} 2023-02-13 22:00:30,474 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-13 22:00:30,474 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 in 182 msec 2023-02-13 22:00:30,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-13 22:00:30,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN in 343 msec 2023-02-13 22:00:30,478 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:00:30,478 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325630478"}]},"ts":"1676325630478"} 2023-02-13 22:00:30,480 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 22:00:30,481 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:00:30,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 387 msec 2023-02-13 22:00:30,613 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:30,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 22:00:30,615 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 22:00:30,615 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,615 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 22:00:30,615 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:00:30,616 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,616 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,616 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,616 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,616 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,616 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,617 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,617 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:30,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:30,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:30,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:30,618 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:30,618 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,618 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:30,618 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:30,618 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:30,618 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,618 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. started... 2023-02-13 22:00:30,619 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 386c949cc4d06cf801b328070dc3f9d1 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 22:00:30,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:30,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:30,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:30,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:30,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:30,621 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:30,621 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,621 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:30,621 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:30,621 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:30,626 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:30,626 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:30,626 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,626 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:30,626 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:30,626 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:30,631 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,633 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/09118292ee6b4757a1b3f3872b9723aa 2023-02-13 22:00:30,637 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,637 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,637 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:30,637 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,637 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:30,646 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/8e32e74b6121419d9382938c4ad55043 2023-02-13 22:00:30,652 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/09118292ee6b4757a1b3f3872b9723aa as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa 2023-02-13 22:00:30,658 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:00:30,659 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/8e32e74b6121419d9382938c4ad55043 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043 2023-02-13 22:00:30,665 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:00:30,666 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 386c949cc4d06cf801b328070dc3f9d1 in 47ms, sequenceid=6, compaction requested=false 2023-02-13 22:00:30,666 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-13 22:00:30,666 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 386c949cc4d06cf801b328070dc3f9d1: 2023-02-13 22:00:30,666 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,666 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:30,666 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:30,667 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,667 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:30,667 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:30,668 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,668 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:30,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:30,668 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,668 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:30,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:30,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:30,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:30,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 22:00:30,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34875aa7[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:30,670 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:00:30,671 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,671 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,671 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,671 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,671 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:30,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,672 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,672 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:30,672 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:30,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,672 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:30,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,672 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:30,672 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:30,672 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:30,672 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:30,672 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:30,672 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:30,673 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:30,673 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:30,673 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:30,673 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:30,673 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:30,673 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:30,673 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,673 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:30,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:30,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:30,673 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:30,674 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:30,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:30,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:30,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:30,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:30,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:00:30,676 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 22:00:30,676 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:30,676 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:30,676 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 22:00:30,676 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:30,677 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,677 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,677 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,677 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,677 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,677 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,677 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:30,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,678 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,679 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,679 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:30,680 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,680 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,680 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,682 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,682 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,682 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,682 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,682 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,682 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:30,682 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,682 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 22:00:30,683 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:30,682 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,682 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,683 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,682 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,683 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:30,683 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,683 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,683 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:30,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,683 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,683 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,684 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,684 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,684 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,783 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:30,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:30,801 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:30,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:30,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 22:00:30,803 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 22:00:30,803 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,804 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 22:00:30,804 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:00:30,804 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,804 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,805 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:30,806 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:30,806 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,807 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:30,807 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:30,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:30,807 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:30,808 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,808 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,808 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,808 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:30,808 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,808 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:30,814 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:30,814 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,814 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:30,814 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. started... 2023-02-13 22:00:30,814 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 386c949cc4d06cf801b328070dc3f9d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:30,823 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/3a5740b9205e4c5c8df91ef951e6e7a7 2023-02-13 22:00:30,834 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/1ce7524662da4cffaf559e5268300ede 2023-02-13 22:00:30,839 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/3a5740b9205e4c5c8df91ef951e6e7a7 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7 2023-02-13 22:00:30,844 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:00:30,845 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/1ce7524662da4cffaf559e5268300ede as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede 2023-02-13 22:00:30,850 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:00:30,851 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 386c949cc4d06cf801b328070dc3f9d1 in 37ms, sequenceid=19, compaction requested=false 2023-02-13 22:00:30,851 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 386c949cc4d06cf801b328070dc3f9d1: 2023-02-13 22:00:30,851 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,851 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:30,851 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:30,851 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,851 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:30,851 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:30,852 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,852 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:30,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:30,853 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,853 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:30,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:30,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:30,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:30,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 22:00:30,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@590cbea2[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:30,854 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:00:30,854 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,855 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,855 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,855 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,855 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:30,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,855 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,855 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:30,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,855 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:30,856 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:30,856 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:30,855 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:30,856 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:30,856 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:30,856 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:30,856 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:30,857 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:30,857 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:30,857 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:30,857 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:30,858 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:30,858 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:30,858 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,858 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:30,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:30,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:30,858 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:30,858 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:30,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:30,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:30,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:30,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:30,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:00:30,860 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 22:00:30,860 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:30,861 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:30,861 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 22:00:30,861 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:30,861 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,861 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,861 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,862 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,862 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,862 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,862 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,862 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:30,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:30,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,862 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:30,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,863 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:30,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,864 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:30,864 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:30,865 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,866 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,866 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,866 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,866 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,866 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 22:00:30,866 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:30,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,866 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,866 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:30,866 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,866 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:30,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,866 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,867 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:30,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:30,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:30,867 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,867 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:30,868 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,868 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,868 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,868 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,868 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:30,868 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:30,868 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,868 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,968 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:30,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:30,987 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:30,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:30,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 22:00:30,989 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 22:00:30,989 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,989 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 22:00:30,989 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:00:30,990 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,990 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,990 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,991 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:30,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,991 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:30,991 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:30,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:30,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:30,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:30,992 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:30,992 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:30,992 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:30,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:30,992 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:30,993 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:30,993 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. started... 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:30,993 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:30,993 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 386c949cc4d06cf801b328070dc3f9d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:30,994 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,994 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,994 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,994 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:30,994 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:30,994 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,002 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/845de8467af447fe8d6e1e345dc41892 2023-02-13 22:00:31,019 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/2f95786d5c1f4702b20b1fc1352c4de5 2023-02-13 22:00:31,023 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/845de8467af447fe8d6e1e345dc41892 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892 2023-02-13 22:00:31,028 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:00:31,029 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/2f95786d5c1f4702b20b1fc1352c4de5 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5 2023-02-13 22:00:31,033 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:00:31,034 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 386c949cc4d06cf801b328070dc3f9d1 in 41ms, sequenceid=32, compaction requested=true 2023-02-13 22:00:31,034 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 386c949cc4d06cf801b328070dc3f9d1: 2023-02-13 22:00:31,034 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:31,034 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:31,034 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:31,034 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,034 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:31,034 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:31,035 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,035 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,035 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,035 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 22:00:31,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5066e5f4[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:31,037 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:00:31,037 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,038 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,038 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,038 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,038 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,038 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,038 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,038 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,038 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:31,038 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,038 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:31,038 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:31,038 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,039 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,039 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:31,039 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,039 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,039 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,039 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,039 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,039 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,040 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,040 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,040 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,040 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:31,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:00:31,043 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 22:00:31,043 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:31,043 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:31,043 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 22:00:31,043 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:31,044 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,044 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,044 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,044 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,044 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,044 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,044 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,044 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,045 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,045 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,047 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,047 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,047 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,050 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 22:00:31,050 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:31,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,050 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,051 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:31,051 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:31,051 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,051 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,051 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,051 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,051 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,051 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,051 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,051 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,151 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:31,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:31,174 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:31,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:31,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 22:00:31,176 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 22:00:31,177 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,177 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 22:00:31,177 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:00:31,177 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,177 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,178 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,178 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,178 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,178 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,179 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,179 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,181 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:31,181 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:31,181 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:31,181 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:31,181 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,181 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:31,181 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,181 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:31,182 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:31,182 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:31,182 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:31,182 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. started... 2023-02-13 22:00:31,182 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 386c949cc4d06cf801b328070dc3f9d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:31,182 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,183 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,183 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,183 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,183 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,183 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,192 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/c2127804cebb47fa88d51755b99a5aad 2023-02-13 22:00:31,208 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/6ebc22e4ea1f4552b9661aebe15eb679 2023-02-13 22:00:31,212 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/c2127804cebb47fa88d51755b99a5aad as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad 2023-02-13 22:00:31,216 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:00:31,217 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/6ebc22e4ea1f4552b9661aebe15eb679 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679 2023-02-13 22:00:31,222 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:00:31,224 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 386c949cc4d06cf801b328070dc3f9d1 in 41ms, sequenceid=45, compaction requested=true 2023-02-13 22:00:31,224 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 386c949cc4d06cf801b328070dc3f9d1: 2023-02-13 22:00:31,224 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:31,224 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:31,224 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:31,224 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,224 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:31,224 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:31,225 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,225 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,226 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,226 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 22:00:31,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25f8a31f[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:31,228 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:00:31,228 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,229 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,229 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,229 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,229 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,229 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,229 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:31,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,229 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,229 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:31,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,229 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,229 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,230 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,230 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:31,230 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,230 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:31,230 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,230 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,230 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,230 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,231 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,231 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,231 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,231 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,231 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,231 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:31,233 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:00:31,233 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 22:00:31,233 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:31,233 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:31,234 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 22:00:31,234 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:31,235 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,235 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,235 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,235 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,236 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,236 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,237 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,237 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,238 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,240 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,240 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,240 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,240 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,240 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,240 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,240 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,240 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,240 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 22:00:31,240 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:31,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,240 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,240 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,241 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:31,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,240 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,241 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:31,241 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,241 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,241 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,241 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,241 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,241 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,241 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,341 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:31,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:31,366 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:31,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:31,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 22:00:31,367 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 22:00:31,367 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,367 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 22:00:31,367 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:00:31,368 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,368 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,369 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,369 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,369 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,369 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,369 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,369 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:31,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:31,370 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:31,370 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:31,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:31,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:31,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:31,370 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:31,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,371 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:31,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:31,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:31,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:31,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:31,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:31,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:31,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:31,371 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:31,371 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. started... 2023-02-13 22:00:31,371 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,371 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 386c949cc4d06cf801b328070dc3f9d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:31,372 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,372 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,372 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,372 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,372 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,379 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/2bd71f435db64291b0c4429aa1dba3bb 2023-02-13 22:00:31,393 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/15f14288e6c14fef9364476c86889ff5 2023-02-13 22:00:31,399 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/2bd71f435db64291b0c4429aa1dba3bb as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb 2023-02-13 22:00:31,405 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:00:31,406 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/15f14288e6c14fef9364476c86889ff5 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5 2023-02-13 22:00:31,413 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:00:31,414 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 386c949cc4d06cf801b328070dc3f9d1 in 43ms, sequenceid=58, compaction requested=true 2023-02-13 22:00:31,414 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 386c949cc4d06cf801b328070dc3f9d1: 2023-02-13 22:00:31,414 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:31,414 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:31,414 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:31,414 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,414 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:31,414 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:31,416 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,416 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,416 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,416 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 22:00:31,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ba48765[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:31,418 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:00:31,419 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,419 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,419 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,419 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,420 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,420 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,420 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:31,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,420 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,420 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:31,420 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,420 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,420 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:31,420 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,420 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,420 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:31,422 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,422 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,422 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,422 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,422 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,422 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,422 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,422 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,422 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,422 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:31,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:00:31,426 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 22:00:31,426 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:31,426 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:31,426 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 22:00:31,426 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:31,429 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,429 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,429 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,429 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,430 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,429 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,430 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,430 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,431 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,432 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,434 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,435 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,435 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,438 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,438 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,438 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,438 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,438 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,438 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,438 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,438 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,438 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,438 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 22:00:31,439 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:31,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,438 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,438 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,439 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:31,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,439 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:31,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,439 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,439 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,439 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,439 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,439 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,439 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,440 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,539 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:31,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:31,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:31,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:31,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 22:00:31,568 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 22:00:31,568 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,568 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 22:00:31,568 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:00:31,568 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,568 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,569 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,570 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,570 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,570 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,570 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,570 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:31,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 22:00:31,572 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:31,572 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:31,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:31,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:31,572 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:31,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:31,572 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:31,573 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:31,573 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:31,573 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. started... 2023-02-13 22:00:31,573 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 386c949cc4d06cf801b328070dc3f9d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:31,573 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,574 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,574 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,574 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,574 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,574 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,582 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/ef874c1ac9d549119dcd9f1c9dbb681d 2023-02-13 22:00:31,593 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/25576bdb1ba746168700c4e5f6b2103f 2023-02-13 22:00:31,598 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/ef874c1ac9d549119dcd9f1c9dbb681d as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d 2023-02-13 22:00:31,603 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:00:31,604 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/25576bdb1ba746168700c4e5f6b2103f as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f 2023-02-13 22:00:31,609 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:00:31,610 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 386c949cc4d06cf801b328070dc3f9d1 in 37ms, sequenceid=71, compaction requested=true 2023-02-13 22:00:31,610 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 386c949cc4d06cf801b328070dc3f9d1: 2023-02-13 22:00:31,610 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:31,610 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:31,610 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:31,610 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,610 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 22:00:31,610 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 22:00:31,611 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,611 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,612 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,612 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:31,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,613 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 22:00:31,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1761c51d[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:31,614 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:00:31,614 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,614 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,614 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,614 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,614 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,614 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:31,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,614 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,614 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,615 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:31,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,614 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,615 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,615 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:31,615 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:00:31,615 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 22:00:31,615 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:31,615 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,615 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,615 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,615 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,616 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,616 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,616 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,616 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 22:00:31,616 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,616 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 22:00:31,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:00:31,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:00:31,618 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 22:00:31,618 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:31,618 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:31,618 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 22:00:31,618 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:31,619 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,619 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,619 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,620 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,620 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,619 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,619 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:31,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,620 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:31,620 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,621 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:31,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:31,622 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 22:00:31,622 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,622 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,632 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,632 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,632 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,632 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,632 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:31,632 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:31,632 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,632 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,632 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 22:00:31,632 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:31,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,632 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,632 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:31,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,633 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:31,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:31,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:31,633 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:31,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 22:00:31,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:31,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:31,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:31,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:31,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:31,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 22:00:31,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 22:00:31,733 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:31,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:32,001 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 22:00:32,003 DEBUG [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:32,006 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,008 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,008 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,008 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,009 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:32,009 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:32,009 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:32,257 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,257 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:32,257 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:32,257 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:32,257 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-13 22:00:32,257 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:32,257 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:32,258 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:32,258 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:32,258 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:00:32,259 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 22:00:32,260 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:32,260 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:32,261 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:32,261 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:32,261 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:32,262 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:32,262 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:32,262 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:32,263 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:32,263 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:32,263 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:32,263 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:32,264 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:32,264 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,265 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:32,265 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,265 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,265 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,265 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 22:00:32,266 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:32,267 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:32,267 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325662267 2023-02-13 22:00:32,269 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:32,269 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,271 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:32,271 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:32,271 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:32,271 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325692271 2023-02-13 22:00:32,272 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325752272 2023-02-13 22:00:32,272 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:32,272 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:32,275 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:32,275 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:32,275 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:32,522 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,522 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:32,522 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:32,522 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:32,523 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:32,523 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:32,523 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:32,523 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:32,523 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:32,524 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:32,528 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 22:00:32,529 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:32,530 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:32,531 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:32,532 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:32,532 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:32,533 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:32,533 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:32,533 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:32,533 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:32,534 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:32,534 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:32,534 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:32,534 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:32,535 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,535 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:32,535 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,535 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,535 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,535 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:00:32,536 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:32,537 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:32,537 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325662537 2023-02-13 22:00:32,538 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:32,538 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,540 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:32,540 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:32,541 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:32,541 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325692541 2023-02-13 22:00:32,541 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325752541 2023-02-13 22:00:32,541 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:32,541 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:32,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN}] 2023-02-13 22:00:32,543 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:32,543 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:32,543 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:32,791 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,791 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:32,792 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:32,792 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:32,792 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:32,793 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:32,793 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:32,793 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:32,793 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:32,795 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:32,798 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 22:00:32,798 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:32,798 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:32,799 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:32,799 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:32,799 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:32,799 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:32,800 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:32,800 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:32,800 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:32,800 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:32,801 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:32,801 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:32,801 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:32,802 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,802 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:32,802 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,802 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,802 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,802 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 22:00:32,803 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:32,803 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:32,804 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325662804 2023-02-13 22:00:32,804 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:32,805 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:32,807 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:32,807 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:32,808 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:32,808 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325692808 2023-02-13 22:00:32,808 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325752808 2023-02-13 22:00:32,808 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:32,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN}] 2023-02-13 22:00:32,808 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:32,810 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:32,811 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:32,811 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325632811"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325632811"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325632811"}]},"ts":"1676325632811"} 2023-02-13 22:00:32,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:32,812 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:32,812 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:32,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:32,812 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:33,058 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,059 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:33,059 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:33,059 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:33,061 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:33,062 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:33,062 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:33,062 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:33,062 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:33,064 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:33,067 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 22:00:33,067 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 22:00:33,068 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:33,069 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:33,070 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:33,070 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:33,070 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,071 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:33,071 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,071 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,071 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,071 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:33,072 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,072 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,072 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:33,072 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,072 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:33,072 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:33,073 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,073 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,073 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,073 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:33,073 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:33,073 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 22:00:33,074 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:33,074 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:33,074 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1 to restore RIT 2023-02-13 22:00:33,074 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325663074 2023-02-13 22:00:33,075 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:33,075 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:33,077 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:33,077 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSING, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:33,077 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:33,077 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325693077 2023-02-13 22:00:33,077 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325753077 2023-02-13 22:00:33,077 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:33,077 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:33,078 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325633077"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325633077"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325633077"}]},"ts":"1676325633077"} 2023-02-13 22:00:33,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:33,079 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:33,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 386c949cc4d06cf801b328070dc3f9d1, disabling compactions & flushes 2023-02-13 22:00:33,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:33,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:33,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. after waiting 0 ms 2023-02-13 22:00:33,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:33,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 386c949cc4d06cf801b328070dc3f9d1 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 22:00:33,251 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/444c56d43e7640b1afd43efd4c14d49f 2023-02-13 22:00:33,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/af32d0bc54f846ffad31d618d5fa5387 2023-02-13 22:00:33,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/444c56d43e7640b1afd43efd4c14d49f as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f 2023-02-13 22:00:33,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:00:33,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/af32d0bc54f846ffad31d618d5fa5387 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387 2023-02-13 22:00:33,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:00:33,281 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 386c949cc4d06cf801b328070dc3f9d1 in 48ms, sequenceid=82, compaction requested=true 2023-02-13 22:00:33,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 22:00:33,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:33,287 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:33,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 386c949cc4d06cf801b328070dc3f9d1: Waiting for close lock at 1676325633233Running coprocessor pre-close hooks at 1676325633233Disabling compacts and flushes for region at 1676325633233Disabling writes for close at 1676325633233Obtaining lock to block concurrent updates at 1676325633234 (+1 ms)Preparing flush snapshotting stores in 386c949cc4d06cf801b328070dc3f9d1 at 1676325633234Finished memstore snapshotting testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676325633234Flushing stores of testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. at 1676325633236 (+2 ms)Flushing 386c949cc4d06cf801b328070dc3f9d1/cf1: creating writer at 1676325633237 (+1 ms)Flushing 386c949cc4d06cf801b328070dc3f9d1/cf1: appending metadata at 1676325633243 (+6 ms)Flushing 386c949cc4d06cf801b328070dc3f9d1/cf1: closing flushed file at 1676325633243Flushing 386c949cc4d06cf801b328070dc3f9d1/cf2: creating writer at 1676325633256 (+13 ms)Flushing 386c949cc4d06cf801b328070dc3f9d1/cf2: appending metadata at 1676325633257 (+1 ms)Flushing 386c949cc4d06cf801b328070dc3f9d1/cf2: closing flushed file at 1676325633258 (+1 ms)Flushing 386c949cc4d06cf801b328070dc3f9d1/cf1: reopening flushed file at 1676325633269 (+11 ms)Flushing 386c949cc4d06cf801b328070dc3f9d1/cf2: reopening flushed file at 1676325633275 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 386c949cc4d06cf801b328070dc3f9d1 in 48ms, sequenceid=82, compaction requested=true at 1676325633281 (+6 ms)Writing region close event to WAL at 1676325633283 (+2 ms)Running coprocessor post-close hooks at 1676325633286 (+3 ms)Closed at 1676325633287 (+1 ms) 2023-02-13 22:00:33,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,289 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED 2023-02-13 22:00:33,289 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325633289"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325633289"}]},"ts":"1676325633289"} 2023-02-13 22:00:33,290 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:33,290 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:33,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:33,290 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:33,328 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,328 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:33,328 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:33,328 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:33,328 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:33,328 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:33,329 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:33,329 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:33,329 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:33,330 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:00:33,331 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 22:00:33,331 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 22:00:33,332 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 22:00:33,332 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:33,332 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:33,333 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:33,333 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:33,333 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,334 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:33,334 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,334 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,335 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,335 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:33,336 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,336 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,336 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:33,338 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,338 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:33,338 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:33,339 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:33,339 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,339 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,339 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,339 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:33,339 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:33,339 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:33,340 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 22:00:33,340 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:33,341 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:33,341 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1 to restore RIT 2023-02-13 22:00:33,341 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325663341 2023-02-13 22:00:33,342 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:33,342 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:33,344 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:33,345 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:33,345 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:33,345 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325693345 2023-02-13 22:00:33,345 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325753345 2023-02-13 22:00:33,345 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:33,345 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:33,345 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325633345"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325633345"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325633345"}]},"ts":"1676325633345"} 2023-02-13 22:00:33,347 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:33,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-13 22:00:33,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 in 268 msec 2023-02-13 22:00:33,350 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:33,350 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:33,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:33,350 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:33,595 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,596 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:33,596 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:33,596 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:33,598 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:33,598 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:33,598 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:33,599 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:33,599 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:33,600 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:33,604 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 22:00:33,604 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 22:00:33,604 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:33,606 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:33,607 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:33,607 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:33,607 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:33,608 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,608 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:33,609 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,609 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,609 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,610 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:33,610 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,610 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,611 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:33,611 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,612 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:33,612 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:33,612 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:33,612 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,612 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,612 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,612 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:33,612 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:33,612 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-13 22:00:33,613 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:33,614 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:33,614 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1 to restore RIT 2023-02-13 22:00:33,614 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325663614 2023-02-13 22:00:33,614 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:33,615 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:33,617 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:33,617 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:33,617 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:33,617 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325693617 2023-02-13 22:00:33,617 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325753617 2023-02-13 22:00:33,617 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:33,617 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:33,619 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:33,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-13 22:00:33,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN in 811 msec 2023-02-13 22:00:33,624 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:33,624 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:33,624 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:33,867 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,868 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:33,868 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:33,868 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:33,870 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:33,870 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:33,870 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:33,871 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:33,871 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:33,872 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:00:33,873 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 22:00:33,873 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:33,873 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:33,874 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:33,874 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:33,875 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:33,875 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:33,875 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,876 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:33,876 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,877 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:33,877 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,878 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:33,878 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,878 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:33,879 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:33,879 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,880 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:33,880 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:33,880 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:33,880 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:33,880 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:33,880 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 22:00:33,881 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:33,881 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:33,882 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325663882 2023-02-13 22:00:33,882 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:33,883 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,885 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:33,885 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:33,885 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:33,885 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325693885 2023-02-13 22:00:33,886 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325753886 2023-02-13 22:00:33,886 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:33,886 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,886 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,886 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:33,890 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:33,899 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=386c949cc4d06cf801b328070dc3f9d1, threads=14 2023-02-13 22:00:33,900 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,900 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,900 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,901 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,901 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,901 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,901 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,901 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,901 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,901 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,901 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,902 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,902 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:33,902 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,035 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,040 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,040 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,043 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,045 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,051 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,059 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,059 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,070 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,073 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,074 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,075 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,463 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,467 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,468 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 386c949cc4d06cf801b328070dc3f9d1 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 22:00:34,518 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:34,518 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:34,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:34,518 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:34,564 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 22:00:34,636 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:34,636 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:34,636 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:34,636 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:34,637 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:34,637 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:34,637 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:34,637 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:34,637 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:34,637 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:00:34,638 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 22:00:34,638 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:34,638 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:34,639 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:34,639 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:34,639 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:34,639 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:34,640 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:34,640 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:34,640 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:34,641 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:34,641 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:34,641 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:34,641 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:34,642 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:34,642 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:34,642 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:34,643 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:34,643 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:34,643 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:34,643 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:34,643 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:34,643 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:34,643 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:34,643 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:34,643 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 22:00:34,644 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:34,645 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:34,645 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325664645 2023-02-13 22:00:34,646 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:34,647 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:34,649 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:34,649 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:34,649 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:34,649 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325694649 2023-02-13 22:00:34,649 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325754649 2023-02-13 22:00:34,649 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:34,658 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=386c949cc4d06cf801b328070dc3f9d1, threads=14 2023-02-13 22:00:34,659 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,660 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,660 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,660 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,661 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,663 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,663 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,667 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,668 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,668 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,668 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,668 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,671 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,671 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,835 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,846 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,848 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,864 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,886 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,886 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,892 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,898 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,911 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,911 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,915 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,919 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,925 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892 for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,927 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad for region: 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:34,927 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 386c949cc4d06cf801b328070dc3f9d1 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 22:00:34,963 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:34,968 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/eff524ec09e63cfead417cc59f09f92c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:00:34,969 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/a866d0f6ae566c2e8c561a5af50aa06e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:00:34,969 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:34,969 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:34,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:34,969 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:35,150 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,150 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:35,150 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:35,150 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:35,150 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:35,150 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:35,150 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:35,150 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:35,150 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:35,151 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:00:35,152 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 22:00:35,152 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:35,152 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:35,152 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:35,153 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:35,153 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:35,153 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:35,153 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,154 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:35,154 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,154 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,154 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:35,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:35,156 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,156 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:35,156 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:35,157 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:35,158 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,158 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,158 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,158 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:35,158 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:35,158 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 22:00:35,159 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:35,159 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:35,160 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325665160 2023-02-13 22:00:35,161 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:35,161 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,163 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:35,163 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:35,163 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:35,163 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325695163 2023-02-13 22:00:35,163 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325755163 2023-02-13 22:00:35,163 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:35,166 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:35,168 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:35,168 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:35,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:35,414 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,415 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:35,415 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:35,415 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:35,416 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:35,416 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:35,416 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:35,416 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:35,417 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:35,418 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:35,421 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 22:00:35,421 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:35,421 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:35,422 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:35,423 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:35,423 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:35,423 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:35,424 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,425 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:35,425 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,425 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,425 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,426 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:35,426 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,426 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,426 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:35,426 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,427 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:35,427 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:35,427 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:35,427 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,427 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,427 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,427 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:35,427 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:35,427 DEBUG [Listener at localhost.localdomain/42295] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,427 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 22:00:35,428 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:35,428 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:35,429 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325665429 2023-02-13 22:00:35,431 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,431 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:35,433 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:35,433 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:35,433 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:35,433 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325695433 2023-02-13 22:00:35,433 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325755433 2023-02-13 22:00:35,433 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:35,433 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:35,443 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:35,443 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 22:00:35,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:35,443 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:35,685 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,685 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:35,685 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:35,685 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:35,686 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:35,686 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:35,686 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:35,686 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:35,686 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:35,688 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:35,690 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 22:00:35,691 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:35,691 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:35,692 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:35,693 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:35,693 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:35,694 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:35,694 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,695 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:35,695 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,696 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,696 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,697 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:35,697 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,697 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,699 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:35,699 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,699 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:35,699 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:35,699 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:35,700 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,700 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,700 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,700 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:35,700 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:35,700 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-13 22:00:35,700 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:35,701 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:35,701 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325665701 2023-02-13 22:00:35,704 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:35,704 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,706 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:35,706 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:35,706 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:35,706 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325695706 2023-02-13 22:00:35,707 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325755707 2023-02-13 22:00:35,707 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:35,707 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:35,709 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:35,709 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 22:00:35,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:35,709 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:35,959 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,959 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:35,959 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:35,959 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:35,960 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:35,960 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:35,960 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:35,961 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:35,961 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:35,963 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:35,965 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 22:00:35,965 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:35,965 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:35,966 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:35,967 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:35,967 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:35,968 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:35,968 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,969 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:35,969 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,970 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:35,971 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,971 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:35,971 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,971 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:35,971 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:35,972 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,972 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:35,972 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:35,972 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:35,972 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,972 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,972 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,972 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:35,972 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:35,972 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 22:00:35,973 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:35,974 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:35,974 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325665974 2023-02-13 22:00:35,975 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:35,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:35,977 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:35,978 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:35,978 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:35,978 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325695978 2023-02-13 22:00:35,978 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325755978 2023-02-13 22:00:35,978 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:35,978 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:35,980 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:35,981 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 22:00:35,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:35,981 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:36,228 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,228 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:36,228 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:36,229 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:36,229 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:36,229 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:36,229 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:36,229 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:36,229 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:36,230 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:00:36,232 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 22:00:36,232 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:36,232 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:36,233 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:36,233 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:36,233 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:36,234 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:36,234 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:36,235 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:36,235 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:36,235 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:36,236 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:36,236 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:36,237 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:36,237 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:36,237 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:36,238 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,239 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:36,239 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:36,239 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:36,239 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,239 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,239 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,239 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:36,239 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:36,239 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 22:00:36,240 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:36,240 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:36,241 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325666241 2023-02-13 22:00:36,242 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:36,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,244 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:36,244 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:36,244 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:36,244 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325696244 2023-02-13 22:00:36,244 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325756244 2023-02-13 22:00:36,244 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:36,244 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:36,248 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN] 2023-02-13 22:00:36,248 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:36,248 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 22:00:36,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:36,248 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:36,495 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,495 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:36,495 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:36,495 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:36,496 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:36,496 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:36,496 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:36,496 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:36,497 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:36,498 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:36,501 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 22:00:36,501 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:00:36,501 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:00:36,501 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 22:00:36,502 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:36,503 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:36,503 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:36,504 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:36,504 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:36,505 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:36,506 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:36,506 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:36,507 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:36,507 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:36,507 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:36,507 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:36,507 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:36,508 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,508 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:36,508 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:36,508 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:36,508 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:36,508 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,508 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,508 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,509 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:36,509 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:36,509 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:36,509 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:00:36,509 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:36,510 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:36,510 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1 to restore RIT 2023-02-13 22:00:36,510 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325666510 2023-02-13 22:00:36,512 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:36,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,513 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:36,514 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:36,515 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:36,515 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:36,515 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325696515 2023-02-13 22:00:36,515 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325756515 2023-02-13 22:00:36,515 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:36,515 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:36,520 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:36,520 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 22:00:36,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:36,520 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:36,765 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,765 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:36,766 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:36,766 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:36,766 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:36,767 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:36,767 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:36,767 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:36,767 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:36,769 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:36,771 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 22:00:36,771 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 22:00:36,772 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:36,773 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:36,773 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:36,773 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:36,774 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:36,775 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:36,775 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:36,776 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:36,776 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:36,777 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:36,777 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:36,778 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:36,779 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:36,779 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,779 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:36,779 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:36,779 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,779 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,779 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,779 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:36,779 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:00:36,780 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:36,781 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:36,781 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1 to restore RIT 2023-02-13 22:00:36,781 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325666781 2023-02-13 22:00:36,782 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:36,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:36,783 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:36,785 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:36,785 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:36,785 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:36,785 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325696785 2023-02-13 22:00:36,785 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325756785 2023-02-13 22:00:36,785 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:36,785 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:36,788 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:36,788 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 22:00:36,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:36,788 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:37,036 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,036 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:37,036 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:37,036 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:37,037 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:37,037 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:37,037 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:37,038 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:37,038 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:37,040 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:37,042 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 22:00:37,042 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 22:00:37,043 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:37,044 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:37,044 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:37,044 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:37,045 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:37,046 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:37,047 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:37,047 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:37,048 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:37,049 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:37,049 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:37,050 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:37,050 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:37,051 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,051 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:37,051 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:37,051 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,051 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,051 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,051 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:37,051 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-13 22:00:37,052 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:37,053 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:37,053 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1 to restore RIT 2023-02-13 22:00:37,053 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325667053 2023-02-13 22:00:37,053 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:37,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,055 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:37,056 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:37,057 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:37,057 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:37,057 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325697057 2023-02-13 22:00:37,057 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325757057 2023-02-13 22:00:37,057 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:00:37,057 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:37,059 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:37,059 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 22:00:37,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:37,059 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:37,307 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,307 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:37,308 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:37,308 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:37,308 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:37,309 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:37,309 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:37,309 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:37,309 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:37,311 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:37,314 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 22:00:37,314 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 22:00:37,315 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:37,316 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:37,316 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:37,317 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:37,317 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:37,318 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:37,318 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:37,318 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:37,318 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:37,318 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:37,318 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:37,318 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:37,319 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:37,319 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,319 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:37,319 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:37,319 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,319 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,319 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,319 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:00:37,319 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 22:00:37,320 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:37,321 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:37,321 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1 to restore RIT 2023-02-13 22:00:37,321 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325667321 2023-02-13 22:00:37,321 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:37,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,325 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:37,326 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:37,326 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:37,326 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:37,327 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325697326 2023-02-13 22:00:37,327 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325757327 2023-02-13 22:00:37,327 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 22:00:37,327 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:00:37,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e exec-time=5.3230 sec 2023-02-13 22:00:37,330 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:37,330 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=true 2023-02-13 22:00:37,481 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:37,482 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:37,482 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325637482"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325637482"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325637482"}]},"ts":"1676325637482"} 2023-02-13 22:00:37,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:37,486 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:00:37,486 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:37,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:37,486 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:00:37,577 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:37,577 WARN [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-13 22:00:37,577 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:00:37,577 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:37,577 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:37,578 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:37,578 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:00:37,578 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:00:37,578 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:00:37,579 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:37,580 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:00:37,582 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-13 22:00:37,583 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:37,583 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:37,584 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:37,584 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:37,585 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:37,585 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:00:37,586 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:37,586 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:00:37,587 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:37,587 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:00:37,588 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:37,588 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:00:37,589 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:37,589 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:00:37,590 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:37,590 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:00:37,591 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:37,591 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:00:37,591 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:37,592 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:00:37,592 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1 to restore RIT 2023-02-13 22:00:37,592 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325667592 2023-02-13 22:00:37,593 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:37,593 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:00:37,595 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:00:37,595 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=386c949cc4d06cf801b328070dc3f9d1, regionState=OPENING, lastHost=jenkins-hbase20.apache.org,33193,1676325615429, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429, openSeqNum=2 2023-02-13 22:00:37,595 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:37,595 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325697595 2023-02-13 22:00:37,595 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325757595 2023-02-13 22:00:37,595 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:00:37,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:37,596 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325637595"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325637595"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325637595"}]},"ts":"1676325637595"} 2023-02-13 22:00:37,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:37,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:37,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 386c949cc4d06cf801b328070dc3f9d1, NAME => 'testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:37,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:37,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:37,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:37,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:37,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:37,759 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:37,761 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1 2023-02-13 22:00:37,761 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1 2023-02-13 22:00:37,762 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 386c949cc4d06cf801b328070dc3f9d1 columnFamilyName cf1 2023-02-13 22:00:37,774 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa 2023-02-13 22:00:37,778 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb 2023-02-13 22:00:37,781 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7 2023-02-13 22:00:37,785 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f 2023-02-13 22:00:37,788 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892 2023-02-13 22:00:37,791 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad 2023-02-13 22:00:37,794 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d 2023-02-13 22:00:37,794 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(310): Store=386c949cc4d06cf801b328070dc3f9d1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:37,794 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:37,795 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2 2023-02-13 22:00:37,795 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2 2023-02-13 22:00:37,795 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 386c949cc4d06cf801b328070dc3f9d1 columnFamilyName cf2 2023-02-13 22:00:37,801 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5 2023-02-13 22:00:37,805 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede 2023-02-13 22:00:37,808 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f 2023-02-13 22:00:37,812 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5 2023-02-13 22:00:37,816 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679 2023-02-13 22:00:37,819 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043 2023-02-13 22:00:37,823 DEBUG [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387 2023-02-13 22:00:37,823 INFO [StoreOpener-386c949cc4d06cf801b328070dc3f9d1-1] regionserver.HStore(310): Store=386c949cc4d06cf801b328070dc3f9d1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:37,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:37,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:37,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.splits 2023-02-13 22:00:37,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:37,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:37,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 386c949cc4d06cf801b328070dc3f9d1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69402107, jitterRate=0.03417198359966278}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:37,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 386c949cc4d06cf801b328070dc3f9d1: Running coprocessor pre-open hook at 1676325637756Writing region info on filesystem at 1676325637756Initializing all the Stores at 1676325637758 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325637758Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325637759 (+1 ms)Cleaning up temporary data from old regions at 1676325637826 (+67 ms)Cleaning up detritus from prior splits at 1676325637826Running coprocessor post-open hooks at 1676325637829 (+3 ms)Region opened successfully at 1676325637829 2023-02-13 22:00:37,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1., pid=48, masterSystemTime=1676325637749 2023-02-13 22:00:37,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:37,830 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:37,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:37,830 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:37,831 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 22:00:37,831 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 22:00:37,831 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1912): 386c949cc4d06cf801b328070dc3f9d1/cf1 is initiating minor compaction (all files) 2023-02-13 22:00:37,831 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1912): 386c949cc4d06cf801b328070dc3f9d1/cf2 is initiating minor compaction (all files) 2023-02-13 22:00:37,831 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 386c949cc4d06cf801b328070dc3f9d1/cf1 in testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:37,831 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 386c949cc4d06cf801b328070dc3f9d1/cf2 in testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:37,831 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp, totalSize=34.9 K 2023-02-13 22:00:37,831 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp, totalSize=34.9 K 2023-02-13 22:00:37,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:37,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:37,833 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 09118292ee6b4757a1b3f3872b9723aa, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325630602 2023-02-13 22:00:37,833 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 8e32e74b6121419d9382938c4ad55043, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325630602 2023-02-13 22:00:37,833 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:37,833 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 3a5740b9205e4c5c8df91ef951e6e7a7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325630785 2023-02-13 22:00:37,833 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 1ce7524662da4cffaf559e5268300ede, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325630785 2023-02-13 22:00:37,833 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325637833"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325637833"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325637833"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325637833"}]},"ts":"1676325637833"} 2023-02-13 22:00:37,833 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 845de8467af447fe8d6e1e345dc41892, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325630969 2023-02-13 22:00:37,833 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 2f95786d5c1f4702b20b1fc1352c4de5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325630969 2023-02-13 22:00:37,834 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 6ebc22e4ea1f4552b9661aebe15eb679, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325631153 2023-02-13 22:00:37,834 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting c2127804cebb47fa88d51755b99a5aad, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325631153 2023-02-13 22:00:37,834 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 15f14288e6c14fef9364476c86889ff5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325631346 2023-02-13 22:00:37,834 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 2bd71f435db64291b0c4429aa1dba3bb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325631346 2023-02-13 22:00:37,834 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting 25576bdb1ba746168700c4e5f6b2103f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325631543 2023-02-13 22:00:37,835 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting ef874c1ac9d549119dcd9f1c9dbb681d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325631543 2023-02-13 22:00:37,835 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting af32d0bc54f846ffad31d618d5fa5387, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325631735 2023-02-13 22:00:37,835 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting 444c56d43e7640b1afd43efd4c14d49f, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325631735 2023-02-13 22:00:37,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-13 22:00:37,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 in 239 msec 2023-02-13 22:00:37,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN in 1.5920 sec 2023-02-13 22:00:37,846 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-13 22:00:37,849 DEBUG [Listener at localhost.localdomain/42295] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-13 22:00:37,851 INFO [Listener at localhost.localdomain/42295] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:37,853 INFO [Listener at localhost.localdomain/42295] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-13 22:00:37,854 INFO [Listener at localhost.localdomain/42295] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:37,854 INFO [Listener at localhost.localdomain/42295] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-13 22:00:37,861 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] throttle.PressureAwareThroughputController(145): 386c949cc4d06cf801b328070dc3f9d1#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:37,861 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] throttle.PressureAwareThroughputController(145): 386c949cc4d06cf801b328070dc3f9d1#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:37,885 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf1/6889adc6e56b417ea5a9d19fd1c58faa as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/6889adc6e56b417ea5a9d19fd1c58faa 2023-02-13 22:00:37,887 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/.tmp/cf2/3900ffd8ade74f7f911fa477d4940f02 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/3900ffd8ade74f7f911fa477d4940f02 2023-02-13 22:00:37,894 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 386c949cc4d06cf801b328070dc3f9d1/cf2 of 386c949cc4d06cf801b328070dc3f9d1 into 3900ffd8ade74f7f911fa477d4940f02(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:37,894 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 386c949cc4d06cf801b328070dc3f9d1: 2023-02-13 22:00:37,894 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1., storeName=386c949cc4d06cf801b328070dc3f9d1/cf2, priority=9, startTime=1676325637830; duration=0sec 2023-02-13 22:00:37,894 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 386c949cc4d06cf801b328070dc3f9d1/cf1 of 386c949cc4d06cf801b328070dc3f9d1 into 6889adc6e56b417ea5a9d19fd1c58faa(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:37,895 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 386c949cc4d06cf801b328070dc3f9d1: 2023-02-13 22:00:37,895 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:37,895 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1., storeName=386c949cc4d06cf801b328070dc3f9d1/cf1, priority=9, startTime=1676325637829; duration=0sec 2023-02-13 22:00:37,895 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:37,904 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:37,906 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-13 22:00:37,906 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testRollbackAndDoubleExecution 2023-02-13 22:00:37,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:37,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 22:00:37,910 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325637909"}]},"ts":"1676325637909"} 2023-02-13 22:00:37,910 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 22:00:37,911 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-13 22:00:37,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN}] 2023-02-13 22:00:37,913 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN 2023-02-13 22:00:37,913 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:37,913 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325637913"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325637913"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325637913"}]},"ts":"1676325637913"} 2023-02-13 22:00:37,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:38,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 22:00:38,067 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:38,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 386c949cc4d06cf801b328070dc3f9d1, disabling compactions & flushes 2023-02-13 22:00:38,067 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:38,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:38,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. after waiting 0 ms 2023-02-13 22:00:38,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:38,068 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f] to archive 2023-02-13 22:00:38,069 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:38,071 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/09118292ee6b4757a1b3f3872b9723aa 2023-02-13 22:00:38,073 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/3a5740b9205e4c5c8df91ef951e6e7a7 2023-02-13 22:00:38,075 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/845de8467af447fe8d6e1e345dc41892 2023-02-13 22:00:38,077 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/c2127804cebb47fa88d51755b99a5aad 2023-02-13 22:00:38,080 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/2bd71f435db64291b0c4429aa1dba3bb 2023-02-13 22:00:38,082 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/ef874c1ac9d549119dcd9f1c9dbb681d 2023-02-13 22:00:38,084 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/444c56d43e7640b1afd43efd4c14d49f 2023-02-13 22:00:38,090 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387] to archive 2023-02-13 22:00:38,091 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:38,092 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/8e32e74b6121419d9382938c4ad55043 2023-02-13 22:00:38,093 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/1ce7524662da4cffaf559e5268300ede 2023-02-13 22:00:38,094 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/2f95786d5c1f4702b20b1fc1352c4de5 2023-02-13 22:00:38,095 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/6ebc22e4ea1f4552b9661aebe15eb679 2023-02-13 22:00:38,097 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/15f14288e6c14fef9364476c86889ff5 2023-02-13 22:00:38,098 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/25576bdb1ba746168700c4e5f6b2103f 2023-02-13 22:00:38,099 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/af32d0bc54f846ffad31d618d5fa5387 2023-02-13 22:00:38,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 22:00:38,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:38,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1. 2023-02-13 22:00:38,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 386c949cc4d06cf801b328070dc3f9d1: Waiting for close lock at 1676325638067Running coprocessor pre-close hooks at 1676325638067Disabling compacts and flushes for region at 1676325638067Disabling writes for close at 1676325638067Writing region close event to WAL at 1676325638101 (+34 ms)Running coprocessor post-close hooks at 1676325638105 (+4 ms)Closed at 1676325638105 2023-02-13 22:00:38,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:38,107 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=386c949cc4d06cf801b328070dc3f9d1, regionState=CLOSED 2023-02-13 22:00:38,108 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325638107"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325638107"}]},"ts":"1676325638107"} 2023-02-13 22:00:38,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-13 22:00:38,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 386c949cc4d06cf801b328070dc3f9d1, server=jenkins-hbase20.apache.org,33193,1676325615429 in 194 msec 2023-02-13 22:00:38,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-13 22:00:38,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, UNASSIGN in 200 msec 2023-02-13 22:00:38,114 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325638113"}]},"ts":"1676325638113"} 2023-02-13 22:00:38,114 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 22:00:38,116 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-13 22:00:38,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 210 msec 2023-02-13 22:00:38,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 22:00:38,215 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-13 22:00:38,218 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testRollbackAndDoubleExecution 2023-02-13 22:00:38,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:38,223 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:38,224 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:38,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 22:00:38,227 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:38,229 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/recovered.edits] 2023-02-13 22:00:38,231 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/6889adc6e56b417ea5a9d19fd1c58faa to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf1/6889adc6e56b417ea5a9d19fd1c58faa 2023-02-13 22:00:38,233 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/3900ffd8ade74f7f911fa477d4940f02 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/cf2/3900ffd8ade74f7f911fa477d4940f02 2023-02-13 22:00:38,236 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/recovered.edits/90.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1/recovered.edits/90.seqid 2023-02-13 22:00:38,236 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackAndDoubleExecution/386c949cc4d06cf801b328070dc3f9d1 2023-02-13 22:00:38,236 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 22:00:38,240 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:38,242 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-13 22:00:38,243 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-13 22:00:38,244 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:38,244 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-13 22:00:38,244 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325638244"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:38,246 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 22:00:38,246 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 386c949cc4d06cf801b328070dc3f9d1, NAME => 'testRollbackAndDoubleExecution,,1676325630094.386c949cc4d06cf801b328070dc3f9d1.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 22:00:38,246 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-13 22:00:38,246 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676325638246"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:38,248 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-13 22:00:38,250 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:00:38,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 32 msec 2023-02-13 22:00:38,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 22:00:38,326 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-13 22:00:38,348 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=396 (was 400), OpenFileDescriptor=850 (was 846) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=258 (was 254) - SystemLoadAverage LEAK? -, ProcessCount=162 (was 164), AvailableMemoryMB=7706 (was 7418) - AvailableMemoryMB LEAK? - 2023-02-13 22:00:38,360 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=396, OpenFileDescriptor=850, MaxFileDescriptor=60000, SystemLoadAverage=258, ProcessCount=162, AvailableMemoryMB=7705 2023-02-13 22:00:38,361 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:38,363 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:00:38,364 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-13 22:00:38,365 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:00:38,367 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:00:38,368 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:00:38,370 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:38,370 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c empty. 2023-02-13 22:00:38,371 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:38,371 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 22:00:38,382 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:38,383 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 40b25f110a291fb9097d057d629f605c, NAME => 'testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:00:38,397 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:38,397 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 40b25f110a291fb9097d057d629f605c, disabling compactions & flushes 2023-02-13 22:00:38,397 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:38,397 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:38,397 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. after waiting 0 ms 2023-02-13 22:00:38,398 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:38,398 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:38,398 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 40b25f110a291fb9097d057d629f605c: Waiting for close lock at 1676325638397Disabling compacts and flushes for region at 1676325638397Disabling writes for close at 1676325638398 (+1 ms)Writing region close event to WAL at 1676325638398Closed at 1676325638398 2023-02-13 22:00:38,400 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:00:38,401 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676325638400"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325638400"}]},"ts":"1676325638400"} 2023-02-13 22:00:38,402 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:00:38,403 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:00:38,403 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325638403"}]},"ts":"1676325638403"} 2023-02-13 22:00:38,404 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-13 22:00:38,406 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:38,407 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:38,407 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:38,407 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:38,407 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:38,407 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=40b25f110a291fb9097d057d629f605c, ASSIGN}] 2023-02-13 22:00:38,409 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=40b25f110a291fb9097d057d629f605c, ASSIGN 2023-02-13 22:00:38,409 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=40b25f110a291fb9097d057d629f605c, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:00:38,560 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:38,562 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=40b25f110a291fb9097d057d629f605c, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:38,562 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676325638562"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325638562"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325638562"}]},"ts":"1676325638562"} 2023-02-13 22:00:38,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 40b25f110a291fb9097d057d629f605c, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:38,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:38,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 40b25f110a291fb9097d057d629f605c, NAME => 'testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:38,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:38,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:38,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:38,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:38,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:38,731 INFO [StoreOpener-40b25f110a291fb9097d057d629f605c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:38,733 DEBUG [StoreOpener-40b25f110a291fb9097d057d629f605c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/cf1 2023-02-13 22:00:38,733 DEBUG [StoreOpener-40b25f110a291fb9097d057d629f605c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/cf1 2023-02-13 22:00:38,733 INFO [StoreOpener-40b25f110a291fb9097d057d629f605c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 40b25f110a291fb9097d057d629f605c columnFamilyName cf1 2023-02-13 22:00:38,734 INFO [StoreOpener-40b25f110a291fb9097d057d629f605c-1] regionserver.HStore(310): Store=40b25f110a291fb9097d057d629f605c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:38,734 INFO [StoreOpener-40b25f110a291fb9097d057d629f605c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:38,736 DEBUG [StoreOpener-40b25f110a291fb9097d057d629f605c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/cf2 2023-02-13 22:00:38,736 DEBUG [StoreOpener-40b25f110a291fb9097d057d629f605c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/cf2 2023-02-13 22:00:38,737 INFO [StoreOpener-40b25f110a291fb9097d057d629f605c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 40b25f110a291fb9097d057d629f605c columnFamilyName cf2 2023-02-13 22:00:38,738 INFO [StoreOpener-40b25f110a291fb9097d057d629f605c-1] regionserver.HStore(310): Store=40b25f110a291fb9097d057d629f605c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:38,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:38,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:38,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:38,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:38,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:38,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 40b25f110a291fb9097d057d629f605c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59307903, jitterRate=-0.116243377327919}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:38,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 40b25f110a291fb9097d057d629f605c: Running coprocessor pre-open hook at 1676325638729Writing region info on filesystem at 1676325638729Initializing all the Stores at 1676325638730 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325638730Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325638731 (+1 ms)Cleaning up temporary data from old regions at 1676325638741 (+10 ms)Cleaning up detritus from prior splits at 1676325638741Running coprocessor post-open hooks at 1676325638747 (+6 ms)Region opened successfully at 1676325638747 2023-02-13 22:00:38,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c., pid=55, masterSystemTime=1676325638720 2023-02-13 22:00:38,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:38,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:38,749 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=40b25f110a291fb9097d057d629f605c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:38,750 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676325638749"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325638749"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325638749"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325638749"}]},"ts":"1676325638749"} 2023-02-13 22:00:38,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-13 22:00:38,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 40b25f110a291fb9097d057d629f605c, server=jenkins-hbase20.apache.org,33193,1676325615429 in 185 msec 2023-02-13 22:00:38,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-13 22:00:38,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=40b25f110a291fb9097d057d629f605c, ASSIGN in 345 msec 2023-02-13 22:00:38,754 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:00:38,754 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325638754"}]},"ts":"1676325638754"} 2023-02-13 22:00:38,755 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-13 22:00:38,756 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:00:38,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 392 msec 2023-02-13 22:00:38,867 DEBUG [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:38,869 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:38,871 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:38,871 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:38,871 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:38,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=40b25f110a291fb9097d057d629f605c, UNASSIGN}] 2023-02-13 22:00:38,877 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=40b25f110a291fb9097d057d629f605c, UNASSIGN 2023-02-13 22:00:38,878 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=40b25f110a291fb9097d057d629f605c, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:38,879 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676325638878"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325638878"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325638878"}]},"ts":"1676325638878"} 2023-02-13 22:00:38,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 40b25f110a291fb9097d057d629f605c, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:39,033 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:39,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 40b25f110a291fb9097d057d629f605c, disabling compactions & flushes 2023-02-13 22:00:39,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:39,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:39,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. after waiting 0 ms 2023-02-13 22:00:39,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:39,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 22:00:39,046 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:39,046 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c. 2023-02-13 22:00:39,046 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 40b25f110a291fb9097d057d629f605c: Waiting for close lock at 1676325639033Running coprocessor pre-close hooks at 1676325639033Disabling compacts and flushes for region at 1676325639033Disabling writes for close at 1676325639036 (+3 ms)Writing region close event to WAL at 1676325639037 (+1 ms)Running coprocessor post-close hooks at 1676325639046 (+9 ms)Closed at 1676325639046 2023-02-13 22:00:39,048 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:39,049 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=40b25f110a291fb9097d057d629f605c, regionState=CLOSED 2023-02-13 22:00:39,049 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676325639049"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325639049"}]},"ts":"1676325639049"} 2023-02-13 22:00:39,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-13 22:00:39,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 40b25f110a291fb9097d057d629f605c, server=jenkins-hbase20.apache.org,33193,1676325615429 in 171 msec 2023-02-13 22:00:39,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-13 22:00:39,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=40b25f110a291fb9097d057d629f605c, UNASSIGN in 178 msec 2023-02-13 22:00:39,079 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 22:00:39,081 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 22:00:39,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676325639082"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676325639082"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676325639082"}]},"ts":"1676325639082"} 2023-02-13 22:00:39,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325639082"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325639082"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325639082"}]},"ts":"1676325639082"} 2023-02-13 22:00:39,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325639082"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325639082"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325639082"}]},"ts":"1676325639082"} 2023-02-13 22:00:39,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dca75ce4d92efcc1c669fe0165d9507, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=61a4873d60328da5b81ad52cecec03ce, ASSIGN}] 2023-02-13 22:00:39,086 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=61a4873d60328da5b81ad52cecec03ce, ASSIGN 2023-02-13 22:00:39,087 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=61a4873d60328da5b81ad52cecec03ce, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:00:39,088 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dca75ce4d92efcc1c669fe0165d9507, ASSIGN 2023-02-13 22:00:39,088 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dca75ce4d92efcc1c669fe0165d9507, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:00:39,237 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 22:00:39,241 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=2dca75ce4d92efcc1c669fe0165d9507, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:39,241 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325639241"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325639241"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325639241"}]},"ts":"1676325639241"} 2023-02-13 22:00:39,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 2dca75ce4d92efcc1c669fe0165d9507, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:39,249 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=61a4873d60328da5b81ad52cecec03ce, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:39,249 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325639249"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325639249"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325639249"}]},"ts":"1676325639249"} 2023-02-13 22:00:39,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 61a4873d60328da5b81ad52cecec03ce, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:39,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce. 2023-02-13 22:00:39,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 61a4873d60328da5b81ad52cecec03ce, NAME => 'testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 22:00:39,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:39,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:39,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,407 INFO [StoreOpener-61a4873d60328da5b81ad52cecec03ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,409 DEBUG [StoreOpener-61a4873d60328da5b81ad52cecec03ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/cf1 2023-02-13 22:00:39,409 DEBUG [StoreOpener-61a4873d60328da5b81ad52cecec03ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/cf1 2023-02-13 22:00:39,410 INFO [StoreOpener-61a4873d60328da5b81ad52cecec03ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 61a4873d60328da5b81ad52cecec03ce columnFamilyName cf1 2023-02-13 22:00:39,411 INFO [StoreOpener-61a4873d60328da5b81ad52cecec03ce-1] regionserver.HStore(310): Store=61a4873d60328da5b81ad52cecec03ce/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:39,411 INFO [StoreOpener-61a4873d60328da5b81ad52cecec03ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,413 DEBUG [StoreOpener-61a4873d60328da5b81ad52cecec03ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/cf2 2023-02-13 22:00:39,413 DEBUG [StoreOpener-61a4873d60328da5b81ad52cecec03ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/cf2 2023-02-13 22:00:39,413 INFO [StoreOpener-61a4873d60328da5b81ad52cecec03ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 61a4873d60328da5b81ad52cecec03ce columnFamilyName cf2 2023-02-13 22:00:39,414 INFO [StoreOpener-61a4873d60328da5b81ad52cecec03ce-1] regionserver.HStore(310): Store=61a4873d60328da5b81ad52cecec03ce/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:39,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:39,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 61a4873d60328da5b81ad52cecec03ce; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67062088, jitterRate=-6.97016716003418E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:39,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 61a4873d60328da5b81ad52cecec03ce: Running coprocessor pre-open hook at 1676325639405Writing region info on filesystem at 1676325639405Initializing all the Stores at 1676325639407 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325639407Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325639407Cleaning up temporary data from old regions at 1676325639418 (+11 ms)Cleaning up detritus from prior splits at 1676325639418Running coprocessor post-open hooks at 1676325639424 (+6 ms)Region opened successfully at 1676325639424 2023-02-13 22:00:39,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce., pid=62, masterSystemTime=1676325639397 2023-02-13 22:00:39,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce. 2023-02-13 22:00:39,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce. 2023-02-13 22:00:39,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507. 2023-02-13 22:00:39,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2dca75ce4d92efcc1c669fe0165d9507, NAME => 'testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 22:00:39,427 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=61a4873d60328da5b81ad52cecec03ce, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:39,428 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325639427"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325639427"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325639427"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325639427"}]},"ts":"1676325639427"} 2023-02-13 22:00:39,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:39,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:39,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,429 INFO [StoreOpener-2dca75ce4d92efcc1c669fe0165d9507-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,431 DEBUG [StoreOpener-2dca75ce4d92efcc1c669fe0165d9507-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/cf1 2023-02-13 22:00:39,431 DEBUG [StoreOpener-2dca75ce4d92efcc1c669fe0165d9507-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/cf1 2023-02-13 22:00:39,431 INFO [StoreOpener-2dca75ce4d92efcc1c669fe0165d9507-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2dca75ce4d92efcc1c669fe0165d9507 columnFamilyName cf1 2023-02-13 22:00:39,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-13 22:00:39,432 INFO [StoreOpener-2dca75ce4d92efcc1c669fe0165d9507-1] regionserver.HStore(310): Store=2dca75ce4d92efcc1c669fe0165d9507/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:39,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 61a4873d60328da5b81ad52cecec03ce, server=jenkins-hbase20.apache.org,33193,1676325615429 in 179 msec 2023-02-13 22:00:39,432 INFO [StoreOpener-2dca75ce4d92efcc1c669fe0165d9507-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=61a4873d60328da5b81ad52cecec03ce, ASSIGN in 346 msec 2023-02-13 22:00:39,433 DEBUG [StoreOpener-2dca75ce4d92efcc1c669fe0165d9507-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/cf2 2023-02-13 22:00:39,433 DEBUG [StoreOpener-2dca75ce4d92efcc1c669fe0165d9507-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/cf2 2023-02-13 22:00:39,434 INFO [StoreOpener-2dca75ce4d92efcc1c669fe0165d9507-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2dca75ce4d92efcc1c669fe0165d9507 columnFamilyName cf2 2023-02-13 22:00:39,434 INFO [StoreOpener-2dca75ce4d92efcc1c669fe0165d9507-1] regionserver.HStore(310): Store=2dca75ce4d92efcc1c669fe0165d9507/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:39,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:39,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 2dca75ce4d92efcc1c669fe0165d9507; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74381916, jitterRate=0.10837692022323608}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:39,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 2dca75ce4d92efcc1c669fe0165d9507: Running coprocessor pre-open hook at 1676325639428Writing region info on filesystem at 1676325639428Initializing all the Stores at 1676325639429 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325639429Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325639429Cleaning up temporary data from old regions at 1676325639437 (+8 ms)Cleaning up detritus from prior splits at 1676325639437Running coprocessor post-open hooks at 1676325639441 (+4 ms)Region opened successfully at 1676325639441 2023-02-13 22:00:39,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507., pid=61, masterSystemTime=1676325639397 2023-02-13 22:00:39,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507. 2023-02-13 22:00:39,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507. 2023-02-13 22:00:39,443 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=2dca75ce4d92efcc1c669fe0165d9507, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:39,443 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325639443"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325639443"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325639443"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325639443"}]},"ts":"1676325639443"} 2023-02-13 22:00:39,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-13 22:00:39,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 2dca75ce4d92efcc1c669fe0165d9507, server=jenkins-hbase20.apache.org,33193,1676325615429 in 200 msec 2023-02-13 22:00:39,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-13 22:00:39,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dca75ce4d92efcc1c669fe0165d9507, ASSIGN in 360 msec 2023-02-13 22:00:39,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce in 580 msec 2023-02-13 22:00:39,626 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 22:00:39,631 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:39,633 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-13 22:00:39,634 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitTableRegionNoStoreFile 2023-02-13 22:00:39,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:00:39,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 22:00:39,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325639638"}]},"ts":"1676325639638"} 2023-02-13 22:00:39,639 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-13 22:00:39,640 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-13 22:00:39,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dca75ce4d92efcc1c669fe0165d9507, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=61a4873d60328da5b81ad52cecec03ce, UNASSIGN}] 2023-02-13 22:00:39,641 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=61a4873d60328da5b81ad52cecec03ce, UNASSIGN 2023-02-13 22:00:39,642 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=61a4873d60328da5b81ad52cecec03ce, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:39,642 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325639642"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325639642"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325639642"}]},"ts":"1676325639642"} 2023-02-13 22:00:39,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 61a4873d60328da5b81ad52cecec03ce, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:39,645 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dca75ce4d92efcc1c669fe0165d9507, UNASSIGN 2023-02-13 22:00:39,646 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=2dca75ce4d92efcc1c669fe0165d9507, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:39,646 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325639646"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325639646"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325639646"}]},"ts":"1676325639646"} 2023-02-13 22:00:39,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 2dca75ce4d92efcc1c669fe0165d9507, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:00:39,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 22:00:39,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 2dca75ce4d92efcc1c669fe0165d9507, disabling compactions & flushes 2023-02-13 22:00:39,796 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507. 2023-02-13 22:00:39,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507. 2023-02-13 22:00:39,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507. after waiting 0 ms 2023-02-13 22:00:39,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507. 2023-02-13 22:00:39,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 22:00:39,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:39,805 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507. 2023-02-13 22:00:39,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 2dca75ce4d92efcc1c669fe0165d9507: Waiting for close lock at 1676325639795Running coprocessor pre-close hooks at 1676325639795Disabling compacts and flushes for region at 1676325639795Disabling writes for close at 1676325639797 (+2 ms)Writing region close event to WAL at 1676325639797Running coprocessor post-close hooks at 1676325639805 (+8 ms)Closed at 1676325639805 2023-02-13 22:00:39,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 61a4873d60328da5b81ad52cecec03ce, disabling compactions & flushes 2023-02-13 22:00:39,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce. 2023-02-13 22:00:39,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce. 2023-02-13 22:00:39,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce. after waiting 0 ms 2023-02-13 22:00:39,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce. 2023-02-13 22:00:39,809 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=2dca75ce4d92efcc1c669fe0165d9507, regionState=CLOSED 2023-02-13 22:00:39,809 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325639809"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325639809"}]},"ts":"1676325639809"} 2023-02-13 22:00:39,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 22:00:39,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:39,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce. 2023-02-13 22:00:39,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 61a4873d60328da5b81ad52cecec03ce: Waiting for close lock at 1676325639807Running coprocessor pre-close hooks at 1676325639807Disabling compacts and flushes for region at 1676325639807Disabling writes for close at 1676325639809 (+2 ms)Writing region close event to WAL at 1676325639809Running coprocessor post-close hooks at 1676325639812 (+3 ms)Closed at 1676325639812 2023-02-13 22:00:39,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-13 22:00:39,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 2dca75ce4d92efcc1c669fe0165d9507, server=jenkins-hbase20.apache.org,33193,1676325615429 in 163 msec 2023-02-13 22:00:39,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=61a4873d60328da5b81ad52cecec03ce, regionState=CLOSED 2023-02-13 22:00:39,816 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325639815"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325639815"}]},"ts":"1676325639815"} 2023-02-13 22:00:39,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-13 22:00:39,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 61a4873d60328da5b81ad52cecec03ce, server=jenkins-hbase20.apache.org,33193,1676325615429 in 173 msec 2023-02-13 22:00:39,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=61a4873d60328da5b81ad52cecec03ce, UNASSIGN in 178 msec 2023-02-13 22:00:39,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-13 22:00:39,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dca75ce4d92efcc1c669fe0165d9507, UNASSIGN in 179 msec 2023-02-13 22:00:39,821 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325639820"}]},"ts":"1676325639820"} 2023-02-13 22:00:39,822 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-13 22:00:39,823 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-13 22:00:39,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 189 msec 2023-02-13 22:00:39,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 22:00:39,940 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-13 22:00:39,941 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitTableRegionNoStoreFile 2023-02-13 22:00:39,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:00:39,944 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:00:39,945 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:00:39,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 22:00:39,954 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:39,954 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,954 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,957 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/recovered.edits] 2023-02-13 22:00:39,958 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/recovered.edits] 2023-02-13 22:00:39,958 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/recovered.edits] 2023-02-13 22:00:39,967 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/recovered.edits/7.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce/recovered.edits/7.seqid 2023-02-13 22:00:39,967 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/recovered.edits/7.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507/recovered.edits/7.seqid 2023-02-13 22:00:39,967 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/recovered.edits/4.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c/recovered.edits/4.seqid 2023-02-13 22:00:39,967 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:00:39,967 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/2dca75ce4d92efcc1c669fe0165d9507 2023-02-13 22:00:39,967 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionNoStoreFile/40b25f110a291fb9097d057d629f605c 2023-02-13 22:00:39,968 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 22:00:39,969 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:00:39,970 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-13 22:00:39,972 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-13 22:00:39,973 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:00:39,973 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-13 22:00:39,973 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325639973"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:39,973 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325639973"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:39,974 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325639973"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:39,975 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 22:00:39,975 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 40b25f110a291fb9097d057d629f605c, NAME => 'testSplitTableRegionNoStoreFile,,1676325638364.40b25f110a291fb9097d057d629f605c.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 2dca75ce4d92efcc1c669fe0165d9507, NAME => 'testSplitTableRegionNoStoreFile,,1676325638867.2dca75ce4d92efcc1c669fe0165d9507.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 61a4873d60328da5b81ad52cecec03ce, NAME => 'testSplitTableRegionNoStoreFile,41,1676325638867.61a4873d60328da5b81ad52cecec03ce.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 22:00:39,975 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-13 22:00:39,975 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676325639975"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:39,977 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-13 22:00:39,978 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:00:39,979 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 37 msec 2023-02-13 22:00:40,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 22:00:40,048 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-13 22:00:40,070 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=400 (was 396) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_606012450_15 at /127.0.0.1:54360 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_606012450_15 at /127.0.0.1:43124 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=850 (was 850), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=269 (was 258) - SystemLoadAverage LEAK? -, ProcessCount=162 (was 162), AvailableMemoryMB=7704 (was 7705) 2023-02-13 22:00:40,085 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=400, OpenFileDescriptor=850, MaxFileDescriptor=60000, SystemLoadAverage=269, ProcessCount=162, AvailableMemoryMB=7703 2023-02-13 22:00:40,086 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:40,088 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:00:40,089 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-13 22:00:40,090 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,091 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:00:40,092 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:00:40,093 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:40,094 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e empty. 2023-02-13 22:00:40,094 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:40,094 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 22:00:40,104 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:40,106 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => b54fd364e5081ef6e29844259f5ed76e, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:00:40,117 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:40,117 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing b54fd364e5081ef6e29844259f5ed76e, disabling compactions & flushes 2023-02-13 22:00:40,117 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,117 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,117 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. after waiting 0 ms 2023-02-13 22:00:40,117 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,117 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,117 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for b54fd364e5081ef6e29844259f5ed76e: Waiting for close lock at 1676325640117Disabling compacts and flushes for region at 1676325640117Disabling writes for close at 1676325640117Writing region close event to WAL at 1676325640117Closed at 1676325640117 2023-02-13 22:00:40,120 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:00:40,121 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676325640121"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325640121"}]},"ts":"1676325640121"} 2023-02-13 22:00:40,123 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:00:40,124 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:00:40,124 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325640124"}]},"ts":"1676325640124"} 2023-02-13 22:00:40,125 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-13 22:00:40,128 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:40,129 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:40,129 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:40,129 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:40,129 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:40,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b54fd364e5081ef6e29844259f5ed76e, ASSIGN}] 2023-02-13 22:00:40,131 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b54fd364e5081ef6e29844259f5ed76e, ASSIGN 2023-02-13 22:00:40,132 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b54fd364e5081ef6e29844259f5ed76e, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37979,1676325615388; forceNewPlan=false, retain=false 2023-02-13 22:00:40,282 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:40,283 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=b54fd364e5081ef6e29844259f5ed76e, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,283 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676325640283"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325640283"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325640283"}]},"ts":"1676325640283"} 2023-02-13 22:00:40,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure b54fd364e5081ef6e29844259f5ed76e, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:00:40,437 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,437 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 22:00:40,440 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:50918, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 22:00:40,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b54fd364e5081ef6e29844259f5ed76e, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:40,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:40,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:40,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:40,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:40,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:40,447 INFO [StoreOpener-b54fd364e5081ef6e29844259f5ed76e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:40,450 DEBUG [StoreOpener-b54fd364e5081ef6e29844259f5ed76e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1 2023-02-13 22:00:40,450 DEBUG [StoreOpener-b54fd364e5081ef6e29844259f5ed76e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1 2023-02-13 22:00:40,450 INFO [StoreOpener-b54fd364e5081ef6e29844259f5ed76e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b54fd364e5081ef6e29844259f5ed76e columnFamilyName cf1 2023-02-13 22:00:40,451 INFO [StoreOpener-b54fd364e5081ef6e29844259f5ed76e-1] regionserver.HStore(310): Store=b54fd364e5081ef6e29844259f5ed76e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:40,451 INFO [StoreOpener-b54fd364e5081ef6e29844259f5ed76e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:40,452 DEBUG [StoreOpener-b54fd364e5081ef6e29844259f5ed76e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2 2023-02-13 22:00:40,452 DEBUG [StoreOpener-b54fd364e5081ef6e29844259f5ed76e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2 2023-02-13 22:00:40,453 INFO [StoreOpener-b54fd364e5081ef6e29844259f5ed76e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b54fd364e5081ef6e29844259f5ed76e columnFamilyName cf2 2023-02-13 22:00:40,453 INFO [StoreOpener-b54fd364e5081ef6e29844259f5ed76e-1] regionserver.HStore(310): Store=b54fd364e5081ef6e29844259f5ed76e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:40,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:40,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:40,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:40,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:40,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:40,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened b54fd364e5081ef6e29844259f5ed76e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62126490, jitterRate=-0.07424315810203552}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:40,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for b54fd364e5081ef6e29844259f5ed76e: Running coprocessor pre-open hook at 1676325640446Writing region info on filesystem at 1676325640446Initializing all the Stores at 1676325640447 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325640447Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325640447Cleaning up temporary data from old regions at 1676325640455 (+8 ms)Cleaning up detritus from prior splits at 1676325640455Running coprocessor post-open hooks at 1676325640461 (+6 ms)Region opened successfully at 1676325640461 2023-02-13 22:00:40,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e., pid=71, masterSystemTime=1676325640437 2023-02-13 22:00:40,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,466 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=b54fd364e5081ef6e29844259f5ed76e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,466 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676325640466"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325640466"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325640466"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325640466"}]},"ts":"1676325640466"} 2023-02-13 22:00:40,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-13 22:00:40,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure b54fd364e5081ef6e29844259f5ed76e, server=jenkins-hbase20.apache.org,37979,1676325615388 in 182 msec 2023-02-13 22:00:40,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-13 22:00:40,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b54fd364e5081ef6e29844259f5ed76e, ASSIGN in 340 msec 2023-02-13 22:00:40,472 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:00:40,472 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325640472"}]},"ts":"1676325640472"} 2023-02-13 22:00:40,473 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-13 22:00:40,475 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:00:40,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 386 msec 2023-02-13 22:00:40,592 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 22:00:40,596 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:50934, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 22:00:40,603 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:40,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,605 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:40,605 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,605 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 22:00:40,605 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 22:00:40,605 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,605 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,606 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,606 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,606 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,606 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,606 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,607 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,607 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:40,607 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,607 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:40,607 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:40,607 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:40,608 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,608 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,608 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:40,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,611 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:40,611 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,611 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:40,614 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,614 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:40,614 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,614 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:40,614 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:40,614 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:40,614 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:40,614 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,614 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:40,615 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. started... 2023-02-13 22:00:40,616 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b54fd364e5081ef6e29844259f5ed76e 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 22:00:40,622 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,623 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,623 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:40,628 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/86e3cd83867740c3801f67d200ecc603 2023-02-13 22:00:40,640 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/283756b61cff410eb263f79d83555dae 2023-02-13 22:00:40,645 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/86e3cd83867740c3801f67d200ecc603 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603 2023-02-13 22:00:40,649 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:00:40,650 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/283756b61cff410eb263f79d83555dae as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae 2023-02-13 22:00:40,656 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:00:40,657 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 42ms, sequenceid=6, compaction requested=false 2023-02-13 22:00:40,658 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:40,659 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b54fd364e5081ef6e29844259f5ed76e: 2023-02-13 22:00:40,659 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,659 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:40,659 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:40,659 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,659 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:40,659 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:40,660 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,660 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:40,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:40,660 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,660 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:40,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:40,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:40,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:40,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 22:00:40,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70cfdb40[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:40,662 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:40,662 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,663 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,663 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,663 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,663 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:40,663 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,663 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:40,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,663 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:40,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,663 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:40,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,663 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:40,663 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:40,663 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:40,663 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:40,663 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:40,663 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:40,664 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,664 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:40,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:40,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:40,664 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:40,664 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:40,664 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:40,664 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:40,664 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:40,664 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:40,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:40,664 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:40,664 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:40,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:40,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:40,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:00:40,667 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:40,667 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 22:00:40,667 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:40,667 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:40,667 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,667 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:40,668 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,668 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:40,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:40,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,668 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,668 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,668 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,668 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,668 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:40,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,669 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,669 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:40,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,669 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:40,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,670 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,670 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,671 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,674 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,674 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:40,674 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:40,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:40,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,674 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,675 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,675 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,675 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,675 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,675 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,775 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:40,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:40,794 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:40,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:40,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,796 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:40,796 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,796 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 22:00:40,796 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 22:00:40,796 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,796 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,797 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,797 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,797 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,797 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,797 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,797 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:40,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:40,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:40,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:40,798 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,798 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:40,798 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,798 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:40,798 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:40,798 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:40,799 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,799 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:40,799 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:40,799 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:40,799 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,799 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:40,799 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:40,799 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. started... 2023-02-13 22:00:40,799 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:40,799 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:40,799 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b54fd364e5081ef6e29844259f5ed76e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:40,800 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,800 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,800 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,800 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,800 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:40,800 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:40,806 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/803226c1943e4bb69ca383613542d1b8 2023-02-13 22:00:40,817 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/ae159d6b94a0429baa84dd1235786f2f 2023-02-13 22:00:40,822 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/803226c1943e4bb69ca383613542d1b8 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8 2023-02-13 22:00:40,827 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:00:40,828 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/ae159d6b94a0429baa84dd1235786f2f as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f 2023-02-13 22:00:40,833 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:00:40,834 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 35ms, sequenceid=19, compaction requested=false 2023-02-13 22:00:40,834 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b54fd364e5081ef6e29844259f5ed76e: 2023-02-13 22:00:40,834 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,834 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:40,835 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:40,835 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,835 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:40,835 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:40,836 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,836 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:40,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:40,837 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,837 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:40,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:40,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:40,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:40,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 22:00:40,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@104f6e1d[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:40,839 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:40,839 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,840 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,840 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,840 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,840 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:40,840 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:40,840 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:40,840 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:40,840 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,840 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:40,840 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:40,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,840 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:40,840 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:40,840 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:40,840 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:40,841 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,841 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:40,841 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:40,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:40,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:40,841 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:40,841 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:40,841 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:40,841 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:40,841 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:40,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:40,842 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:40,842 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:40,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:40,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:40,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:00:40,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:40,845 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 22:00:40,845 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:40,845 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:40,845 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,845 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:40,846 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,846 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,846 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,846 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:40,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:40,846 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,846 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,846 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:40,847 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,847 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:40,848 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,850 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:40,850 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,850 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,856 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,856 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:40,856 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:40,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:40,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,856 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,857 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:40,857 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:40,857 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:40,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:40,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:40,857 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,857 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:40,857 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,857 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,857 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,957 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:40,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:40,985 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:40,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:40,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,988 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:40,988 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,988 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 22:00:40,988 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 22:00:40,988 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,988 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:40,989 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:40,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:40,990 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:40,990 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:40,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:40,991 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:40,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:40,991 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:40,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,991 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:40,992 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:40,992 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:40,992 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:40,991 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:40,992 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. started... 2023-02-13 22:00:40,992 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,992 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:40,992 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:40,992 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b54fd364e5081ef6e29844259f5ed76e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:40,992 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:40,992 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:40,992 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:40,992 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:40,992 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:40,994 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,994 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,994 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,994 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:40,994 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:40,994 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,006 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/58a957378f384351a76d1eafde7ebe1b 2023-02-13 22:00:41,017 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/c2884f31c5ef45a895c688ccc7bbd169 2023-02-13 22:00:41,021 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/58a957378f384351a76d1eafde7ebe1b as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b 2023-02-13 22:00:41,025 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:00:41,026 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/c2884f31c5ef45a895c688ccc7bbd169 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169 2023-02-13 22:00:41,030 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:00:41,031 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 39ms, sequenceid=32, compaction requested=true 2023-02-13 22:00:41,031 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b54fd364e5081ef6e29844259f5ed76e: 2023-02-13 22:00:41,031 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,031 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:41,031 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:41,031 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,031 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,031 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,032 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,032 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,032 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,033 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 22:00:41,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f5f5cde[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:41,034 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:41,034 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,035 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,035 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,035 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,035 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,035 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,035 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,035 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:41,035 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,035 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:41,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,035 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,035 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:41,035 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,035 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,035 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:41,036 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,036 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,036 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,036 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,036 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,036 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,036 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,036 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,036 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,036 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:00:41,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:41,039 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 22:00:41,039 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:41,039 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:41,039 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,039 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:41,040 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,040 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,040 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,040 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,041 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,041 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,042 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,042 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,043 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,056 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,056 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,056 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,056 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:41,056 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,056 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,056 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,056 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,056 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,056 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,056 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,056 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:41,056 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,057 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:41,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,057 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,057 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,057 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,057 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,057 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,057 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,057 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,057 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,157 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:41,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:41,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:41,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:41,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,180 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,180 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,180 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 22:00:41,180 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 22:00:41,180 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,180 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,181 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,181 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,181 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,181 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,181 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,181 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,182 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,182 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:41,183 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:41,183 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,183 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. started... 2023-02-13 22:00:41,183 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b54fd364e5081ef6e29844259f5ed76e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:41,184 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,184 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,184 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,184 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,185 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,185 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,192 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/79457093278d407b9ab5920b7232ec51 2023-02-13 22:00:41,203 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/8769cb8cd52441ddae4cd234c69dc2c9 2023-02-13 22:00:41,207 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/79457093278d407b9ab5920b7232ec51 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/79457093278d407b9ab5920b7232ec51 2023-02-13 22:00:41,211 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/79457093278d407b9ab5920b7232ec51, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:00:41,212 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/8769cb8cd52441ddae4cd234c69dc2c9 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/8769cb8cd52441ddae4cd234c69dc2c9 2023-02-13 22:00:41,216 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/8769cb8cd52441ddae4cd234c69dc2c9, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:00:41,217 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 34ms, sequenceid=45, compaction requested=true 2023-02-13 22:00:41,217 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b54fd364e5081ef6e29844259f5ed76e: 2023-02-13 22:00:41,217 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,217 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:41,217 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:41,217 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,217 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,217 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,218 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,218 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,219 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,219 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 22:00:41,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68f2b05[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:41,221 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:41,221 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,222 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,222 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,222 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,222 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,222 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,222 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:41,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,222 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,222 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:41,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,222 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,222 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,222 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,222 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,222 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:41,222 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:41,223 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,223 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,223 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,223 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,223 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,223 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,223 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,225 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,225 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,225 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:00:41,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:41,228 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 22:00:41,228 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:41,228 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:41,228 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,228 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:41,229 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,230 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,230 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,229 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,230 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,230 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,230 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,230 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,231 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,232 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,234 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,234 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,235 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,238 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,238 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,237 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,238 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,238 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,238 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:41,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,238 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,238 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,238 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,238 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:41,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,238 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,239 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:41,239 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,239 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,239 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,239 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,239 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,339 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:41,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:41,363 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:41,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:41,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,365 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,365 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,365 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 22:00:41,366 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 22:00:41,366 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,366 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,367 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,367 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,367 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,367 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,368 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,368 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:41,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,370 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,370 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,370 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,370 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,371 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,371 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,371 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. started... 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,371 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b54fd364e5081ef6e29844259f5ed76e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:41,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,373 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,374 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,374 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,374 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,374 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,374 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,385 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/4866a8208bd64630b1f438ea502970a4 2023-02-13 22:00:41,395 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/bfc543b7ef0f40dd849ab48173edd3d4 2023-02-13 22:00:41,399 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/4866a8208bd64630b1f438ea502970a4 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/4866a8208bd64630b1f438ea502970a4 2023-02-13 22:00:41,403 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/4866a8208bd64630b1f438ea502970a4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:00:41,404 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/bfc543b7ef0f40dd849ab48173edd3d4 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/bfc543b7ef0f40dd849ab48173edd3d4 2023-02-13 22:00:41,407 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/bfc543b7ef0f40dd849ab48173edd3d4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:00:41,408 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 37ms, sequenceid=58, compaction requested=true 2023-02-13 22:00:41,408 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b54fd364e5081ef6e29844259f5ed76e: 2023-02-13 22:00:41,408 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,408 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:41,408 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:41,408 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,408 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,408 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,409 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,409 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,409 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,409 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 22:00:41,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@894e54f[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:41,412 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:41,412 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,413 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,413 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,413 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,413 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,413 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,413 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:41,413 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,413 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:41,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,413 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,413 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,413 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,414 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:41,413 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,414 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:41,414 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,414 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,414 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,415 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,415 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,415 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,415 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,415 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,415 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,415 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:00:41,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:41,419 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 22:00:41,419 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:41,419 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:41,419 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,419 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:41,420 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,420 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,420 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,420 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,420 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,420 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,421 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,421 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,422 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,422 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,424 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,424 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,424 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,432 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,432 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,432 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,432 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,432 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,432 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,432 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,433 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,432 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,433 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:41,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,432 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,432 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,433 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,433 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:41,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,433 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,433 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:41,433 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,433 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,434 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,434 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,434 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,434 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,534 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:41,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:41,547 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:41,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:41,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,549 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,549 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,549 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 22:00:41,549 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 22:00:41,549 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,549 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,550 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,550 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,550 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,550 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,550 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,550 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:41,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:41,552 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,552 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. started... 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,552 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,552 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b54fd364e5081ef6e29844259f5ed76e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:00:41,553 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,553 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,554 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,554 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,554 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,554 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,565 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/3bb515fe234e430e8ba0e7e988e9d3c1 2023-02-13 22:00:41,576 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/43964f3954e24bde8250ce9a01e9e21f 2023-02-13 22:00:41,580 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/3bb515fe234e430e8ba0e7e988e9d3c1 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3bb515fe234e430e8ba0e7e988e9d3c1 2023-02-13 22:00:41,584 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3bb515fe234e430e8ba0e7e988e9d3c1, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:00:41,584 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/43964f3954e24bde8250ce9a01e9e21f as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/43964f3954e24bde8250ce9a01e9e21f 2023-02-13 22:00:41,588 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/43964f3954e24bde8250ce9a01e9e21f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:00:41,589 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 37ms, sequenceid=71, compaction requested=true 2023-02-13 22:00:41,589 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b54fd364e5081ef6e29844259f5ed76e: 2023-02-13 22:00:41,589 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,589 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:41,589 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:41,589 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,589 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,589 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,591 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,591 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,591 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,591 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,593 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 22:00:41,593 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63c05dc6[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:41,593 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:41,593 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,593 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,593 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,593 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,594 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,594 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,594 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:41,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,594 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,594 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:41,594 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,594 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,594 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:41,594 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,594 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,594 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:41,595 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,595 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,595 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,595 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,595 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,595 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,595 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,595 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,595 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,595 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:00:41,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:41,597 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 22:00:41,597 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:41,597 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:41,598 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,598 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:41,598 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,598 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,598 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,598 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,599 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,599 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,598 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,599 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,599 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,599 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,601 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,601 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,601 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,602 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,602 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,603 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,603 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:41,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,603 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:41,603 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,604 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:41,604 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,604 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,604 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,604 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,604 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,704 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:41,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:41,726 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:41,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:41,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,728 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,728 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,728 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 22:00:41,728 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 22:00:41,729 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,729 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,730 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,730 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,730 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,730 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,730 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,730 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:41,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,732 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,732 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,732 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,732 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,732 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,732 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,732 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,732 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,732 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,732 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,733 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,732 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,733 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,733 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,732 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:41,733 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:41,733 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,733 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,733 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,733 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. started... 2023-02-13 22:00:41,733 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,733 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b54fd364e5081ef6e29844259f5ed76e 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-13 22:00:41,733 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,734 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,734 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,734 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,735 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,735 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,742 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/1db77fb166b947fe82dd19088eda6129 2023-02-13 22:00:41,747 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1db77fb166b947fe82dd19088eda6129 2023-02-13 22:00:41,757 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/7ae1fa8f0b29466988677342f52e7c52 2023-02-13 22:00:41,762 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ae1fa8f0b29466988677342f52e7c52 2023-02-13 22:00:41,763 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/1db77fb166b947fe82dd19088eda6129 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129 2023-02-13 22:00:41,767 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1db77fb166b947fe82dd19088eda6129 2023-02-13 22:00:41,767 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 22:00:41,768 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/7ae1fa8f0b29466988677342f52e7c52 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52 2023-02-13 22:00:41,772 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ae1fa8f0b29466988677342f52e7c52 2023-02-13 22:00:41,772 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 22:00:41,773 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 40ms, sequenceid=83, compaction requested=true 2023-02-13 22:00:41,773 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b54fd364e5081ef6e29844259f5ed76e: 2023-02-13 22:00:41,773 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,773 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:41,773 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:41,773 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,773 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,773 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,775 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,775 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,775 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,775 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 22:00:41,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61c44cc[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:41,777 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:41,778 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,778 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,778 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,778 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,779 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,779 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:41,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,778 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,779 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,779 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,779 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,779 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:41,779 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:41,779 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,779 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,779 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:41,781 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,781 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,781 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,781 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,781 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,781 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,781 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,781 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,781 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,781 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:00:41,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:41,785 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 22:00:41,785 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:41,785 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:41,785 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,785 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:41,795 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,795 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,795 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,795 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,795 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,795 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,796 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,796 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,797 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,798 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,800 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,801 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,801 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,805 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,805 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:41,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,806 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:41,805 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,805 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,806 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:41,806 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,806 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,807 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,807 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,807 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,807 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,807 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,807 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,907 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:41,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:41,925 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:41,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:41,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,926 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,926 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,926 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 22:00:41,926 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 22:00:41,927 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,927 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,927 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,927 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,927 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,927 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,928 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,928 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,928 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,929 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,929 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,929 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:41,929 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:41,929 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. started... 2023-02-13 22:00:41,929 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:41,929 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b54fd364e5081ef6e29844259f5ed76e 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 22:00:41,929 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,930 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,930 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,930 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,930 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,930 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,931 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,931 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,948 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/e3e91e846bf84ab0a0d6a5fc86ab4564 2023-02-13 22:00:41,951 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e3e91e846bf84ab0a0d6a5fc86ab4564 2023-02-13 22:00:41,963 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/1603140b7fe0439bbd846868340b5249 2023-02-13 22:00:41,966 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1603140b7fe0439bbd846868340b5249 2023-02-13 22:00:41,966 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/e3e91e846bf84ab0a0d6a5fc86ab4564 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/e3e91e846bf84ab0a0d6a5fc86ab4564 2023-02-13 22:00:41,970 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e3e91e846bf84ab0a0d6a5fc86ab4564 2023-02-13 22:00:41,970 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/e3e91e846bf84ab0a0d6a5fc86ab4564, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 22:00:41,971 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/1603140b7fe0439bbd846868340b5249 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/1603140b7fe0439bbd846868340b5249 2023-02-13 22:00:41,974 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1603140b7fe0439bbd846868340b5249 2023-02-13 22:00:41,974 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/1603140b7fe0439bbd846868340b5249, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 22:00:41,975 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 46ms, sequenceid=91, compaction requested=true 2023-02-13 22:00:41,975 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b54fd364e5081ef6e29844259f5ed76e: 2023-02-13 22:00:41,975 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:41,975 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:41,975 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:41,975 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,975 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:41,976 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:41,977 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,977 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,977 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,977 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:41,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,978 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 22:00:41,978 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79b49009[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:41,978 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:41,978 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,979 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,979 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,979 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,979 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,979 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,980 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:41,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,980 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,980 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,980 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,980 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:41,980 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:41,980 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:41,980 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:41,980 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:41,980 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,980 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,980 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,981 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,981 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,981 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,981 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,981 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:41,981 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,981 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:41,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:00:41,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:41,983 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 22:00:41,983 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:41,983 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:41,983 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,983 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:41,984 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,984 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,984 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,984 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,984 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:41,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,984 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:41,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,984 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,984 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:41,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,985 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,985 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:41,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,986 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:41,987 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,987 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,989 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:41,989 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:41,990 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:41,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,989 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:41,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:41,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:41,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:41,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:41,990 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,090 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:42,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:42,105 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:00:42,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:00:42,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,108 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:42,108 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:42,108 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 22:00:42,108 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 22:00:42,109 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,109 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,110 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:42,110 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:42,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:42,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:42,110 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,110 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:42,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:42,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:42,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:42,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:42,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,111 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,111 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:00:42,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:42,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:42,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:00:42,112 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:42,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,113 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:42,113 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:42,113 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:42,113 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,113 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:42,113 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:42,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,113 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:42,113 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:42,113 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:42,113 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 22:00:42,113 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:42,114 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:42,114 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:42,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,114 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 22:00:42,114 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:00:42,114 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 22:00:42,114 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:00:42,114 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:00:42,114 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:42,114 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. started... 2023-02-13 22:00:42,115 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b54fd364e5081ef6e29844259f5ed76e 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 22:00:42,115 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,116 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,116 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,116 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:42,116 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,116 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:42,128 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/9cf556e5475141bbb6d23190acae7a51 2023-02-13 22:00:42,134 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9cf556e5475141bbb6d23190acae7a51 2023-02-13 22:00:42,146 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/2aa0cf01906745a1be01e93962540832 2023-02-13 22:00:42,152 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2aa0cf01906745a1be01e93962540832 2023-02-13 22:00:42,153 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/9cf556e5475141bbb6d23190acae7a51 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/9cf556e5475141bbb6d23190acae7a51 2023-02-13 22:00:42,159 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9cf556e5475141bbb6d23190acae7a51 2023-02-13 22:00:42,159 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/9cf556e5475141bbb6d23190acae7a51, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 22:00:42,160 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/2aa0cf01906745a1be01e93962540832 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/2aa0cf01906745a1be01e93962540832 2023-02-13 22:00:42,165 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2aa0cf01906745a1be01e93962540832 2023-02-13 22:00:42,165 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/2aa0cf01906745a1be01e93962540832, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 22:00:42,166 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 51ms, sequenceid=99, compaction requested=true 2023-02-13 22:00:42,166 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b54fd364e5081ef6e29844259f5ed76e: 2023-02-13 22:00:42,166 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:42,166 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:00:42,166 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:00:42,166 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,166 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 22:00:42,166 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 22:00:42,168 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,168 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:42,169 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,169 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:00:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:42,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:42,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:42,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 22:00:42,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56a5bced[Count = 0] remaining members to acquire global barrier 2023-02-13 22:00:42,172 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 22:00:42,172 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,173 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,173 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,173 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,173 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:42,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,173 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,174 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:00:42,173 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:42,174 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:42,174 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:00:42,173 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:42,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,174 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:00:42,174 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 22:00:42,174 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 22:00:42,174 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:00:42,175 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:42,175 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:42,175 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:42,175 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:42,175 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,175 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 22:00:42,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:42,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:42,175 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:42,175 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:42,176 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:42,176 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 22:00:42,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:42,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:42,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:42,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:42,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:42,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:42,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:42,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:00:42,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:00:42,180 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 22:00:42,180 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:00:42,180 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:00:42,180 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,180 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:00:42,182 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,182 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,182 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,182 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:00:42,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,182 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:42,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,182 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:42,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:00:42,182 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:42,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,183 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:42,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:42,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:00:42,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:42,183 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:42,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:42,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,184 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:00:42,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:42,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:42,186 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:00:42,187 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:42,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,187 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:42,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:42,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:42,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:42,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:42,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:42,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:42,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:42,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:42,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:42,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,196 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:42,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:00:42,196 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:00:42,196 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,196 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 22:00:42,196 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:00:42,196 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:00:42,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:42,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:42,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:42,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:42,197 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:00:42,196 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:00:42,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:42,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:42,197 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:00:42,197 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:00:42,197 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:42,197 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:42,197 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,197 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,198 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:42,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:00:42,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:42,297 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:00:42,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:00:42,303 DEBUG [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,304 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:42,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:42,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:42,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:42,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b54fd364e5081ef6e29844259f5ed76e, UNASSIGN}] 2023-02-13 22:00:42,311 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b54fd364e5081ef6e29844259f5ed76e, UNASSIGN 2023-02-13 22:00:42,312 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=b54fd364e5081ef6e29844259f5ed76e, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,312 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676325642312"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325642312"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325642312"}]},"ts":"1676325642312"} 2023-02-13 22:00:42,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure b54fd364e5081ef6e29844259f5ed76e, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:00:42,466 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing b54fd364e5081ef6e29844259f5ed76e, disabling compactions & flushes 2023-02-13 22:00:42,467 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:42,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:42,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. after waiting 0 ms 2023-02-13 22:00:42,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:42,467 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing b54fd364e5081ef6e29844259f5ed76e 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-13 22:00:42,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/3452c0510f184df6bd9669c4f6c64cd0 2023-02-13 22:00:42,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3452c0510f184df6bd9669c4f6c64cd0 2023-02-13 22:00:42,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/f3476e1ed4c84940a753285a6304b428 2023-02-13 22:00:42,510 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f3476e1ed4c84940a753285a6304b428 2023-02-13 22:00:42,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf1/3452c0510f184df6bd9669c4f6c64cd0 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3452c0510f184df6bd9669c4f6c64cd0 2023-02-13 22:00:42,518 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3452c0510f184df6bd9669c4f6c64cd0 2023-02-13 22:00:42,518 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3452c0510f184df6bd9669c4f6c64cd0, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 22:00:42,519 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.tmp/cf2/f3476e1ed4c84940a753285a6304b428 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/f3476e1ed4c84940a753285a6304b428 2023-02-13 22:00:42,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f3476e1ed4c84940a753285a6304b428 2023-02-13 22:00:42,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/f3476e1ed4c84940a753285a6304b428, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 22:00:42,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 58ms, sequenceid=103, compaction requested=true 2023-02-13 22:00:42,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-13 22:00:42,532 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:42,532 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. 2023-02-13 22:00:42,532 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for b54fd364e5081ef6e29844259f5ed76e: Waiting for close lock at 1676325642467Running coprocessor pre-close hooks at 1676325642467Disabling compacts and flushes for region at 1676325642467Disabling writes for close at 1676325642467Obtaining lock to block concurrent updates at 1676325642467Preparing flush snapshotting stores in b54fd364e5081ef6e29844259f5ed76e at 1676325642467Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676325642467Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e. at 1676325642468 (+1 ms)Flushing b54fd364e5081ef6e29844259f5ed76e/cf1: creating writer at 1676325642469 (+1 ms)Flushing b54fd364e5081ef6e29844259f5ed76e/cf1: appending metadata at 1676325642472 (+3 ms)Flushing b54fd364e5081ef6e29844259f5ed76e/cf1: closing flushed file at 1676325642472Flushing b54fd364e5081ef6e29844259f5ed76e/cf2: creating writer at 1676325642492 (+20 ms)Flushing b54fd364e5081ef6e29844259f5ed76e/cf2: appending metadata at 1676325642495 (+3 ms)Flushing b54fd364e5081ef6e29844259f5ed76e/cf2: closing flushed file at 1676325642495Flushing b54fd364e5081ef6e29844259f5ed76e/cf1: reopening flushed file at 1676325642512 (+17 ms)Flushing b54fd364e5081ef6e29844259f5ed76e/cf2: reopening flushed file at 1676325642520 (+8 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for b54fd364e5081ef6e29844259f5ed76e in 58ms, sequenceid=103, compaction requested=true at 1676325642525 (+5 ms)Writing region close event to WAL at 1676325642528 (+3 ms)Running coprocessor post-close hooks at 1676325642532 (+4 ms)Closed at 1676325642532 2023-02-13 22:00:42,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,534 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=b54fd364e5081ef6e29844259f5ed76e, regionState=CLOSED 2023-02-13 22:00:42,534 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676325642534"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325642534"}]},"ts":"1676325642534"} 2023-02-13 22:00:42,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-13 22:00:42,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure b54fd364e5081ef6e29844259f5ed76e, server=jenkins-hbase20.apache.org,37979,1676325615388 in 222 msec 2023-02-13 22:00:42,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-13 22:00:42,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b54fd364e5081ef6e29844259f5ed76e, UNASSIGN in 228 msec 2023-02-13 22:00:42,560 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=b54fd364e5081ef6e29844259f5ed76e, threads=16 2023-02-13 22:00:42,560 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/1603140b7fe0439bbd846868340b5249 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,560 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,560 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/2aa0cf01906745a1be01e93962540832 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,562 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/43964f3954e24bde8250ce9a01e9e21f for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,563 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,563 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/8769cb8cd52441ddae4cd234c69dc2c9 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,566 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,566 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/bfc543b7ef0f40dd849ab48173edd3d4 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,566 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,566 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/f3476e1ed4c84940a753285a6304b428 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,566 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,571 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3452c0510f184df6bd9669c4f6c64cd0 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,571 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3bb515fe234e430e8ba0e7e988e9d3c1 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,571 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/4866a8208bd64630b1f438ea502970a4 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,571 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,571 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/79457093278d407b9ab5920b7232ec51 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,577 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f3476e1ed4c84940a753285a6304b428 2023-02-13 22:00:42,577 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ae1fa8f0b29466988677342f52e7c52 2023-02-13 22:00:42,578 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2aa0cf01906745a1be01e93962540832 2023-02-13 22:00:42,580 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1db77fb166b947fe82dd19088eda6129 2023-02-13 22:00:42,580 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/43964f3954e24bde8250ce9a01e9e21f, top=false 2023-02-13 22:00:42,585 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1603140b7fe0439bbd846868340b5249 2023-02-13 22:00:42,593 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/8769cb8cd52441ddae4cd234c69dc2c9, top=false 2023-02-13 22:00:42,593 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/4866a8208bd64630b1f438ea502970a4, top=false 2023-02-13 22:00:42,628 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f3476e1ed4c84940a753285a6304b428 2023-02-13 22:00:42,628 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/f3476e1ed4c84940a753285a6304b428, top=true 2023-02-13 22:00:42,628 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2aa0cf01906745a1be01e93962540832 2023-02-13 22:00:42,628 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/2aa0cf01906745a1be01e93962540832, top=true 2023-02-13 22:00:42,629 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1603140b7fe0439bbd846868340b5249 2023-02-13 22:00:42,629 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/1603140b7fe0439bbd846868340b5249, top=true 2023-02-13 22:00:42,641 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3bb515fe234e430e8ba0e7e988e9d3c1, top=false 2023-02-13 22:00:42,641 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/79457093278d407b9ab5920b7232ec51, top=false 2023-02-13 22:00:42,642 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3452c0510f184df6bd9669c4f6c64cd0 2023-02-13 22:00:42,642 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/bfc543b7ef0f40dd849ab48173edd3d4, top=false 2023-02-13 22:00:42,644 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ae1fa8f0b29466988677342f52e7c52 2023-02-13 22:00:42,647 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-4866a8208bd64630b1f438ea502970a4 for child: 1405218340a9d9a499f287b55eca8ccc, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,649 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-8769cb8cd52441ddae4cd234c69dc2c9 for child: 1405218340a9d9a499f287b55eca8ccc, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,649 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-43964f3954e24bde8250ce9a01e9e21f for child: 1405218340a9d9a499f287b55eca8ccc, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,652 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3452c0510f184df6bd9669c4f6c64cd0 2023-02-13 22:00:42,652 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3452c0510f184df6bd9669c4f6c64cd0, top=true 2023-02-13 22:00:42,652 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/4866a8208bd64630b1f438ea502970a4 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,653 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,653 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/8769cb8cd52441ddae4cd234c69dc2c9 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,653 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,657 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/43964f3954e24bde8250ce9a01e9e21f for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,657 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/9cf556e5475141bbb6d23190acae7a51 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,663 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-1603140b7fe0439bbd846868340b5249 for child: db1b569d476c9340ddb2430f438d297d, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,663 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/1603140b7fe0439bbd846868340b5249 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,663 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/e3e91e846bf84ab0a0d6a5fc86ab4564 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,669 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9cf556e5475141bbb6d23190acae7a51 2023-02-13 22:00:42,671 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e3e91e846bf84ab0a0d6a5fc86ab4564 2023-02-13 22:00:42,673 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9cf556e5475141bbb6d23190acae7a51 2023-02-13 22:00:42,673 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/9cf556e5475141bbb6d23190acae7a51, top=true 2023-02-13 22:00:42,674 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e3e91e846bf84ab0a0d6a5fc86ab4564 2023-02-13 22:00:42,674 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/e3e91e846bf84ab0a0d6a5fc86ab4564, top=true 2023-02-13 22:00:42,683 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-2aa0cf01906745a1be01e93962540832 for child: db1b569d476c9340ddb2430f438d297d, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,684 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/2aa0cf01906745a1be01e93962540832 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,691 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-f3476e1ed4c84940a753285a6304b428 for child: db1b569d476c9340ddb2430f438d297d, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,691 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/f3476e1ed4c84940a753285a6304b428 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,695 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3bb515fe234e430e8ba0e7e988e9d3c1 for child: 1405218340a9d9a499f287b55eca8ccc, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,711 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3bb515fe234e430e8ba0e7e988e9d3c1 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,715 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-e3e91e846bf84ab0a0d6a5fc86ab4564 for child: db1b569d476c9340ddb2430f438d297d, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,715 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/e3e91e846bf84ab0a0d6a5fc86ab4564 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,717 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-bfc543b7ef0f40dd849ab48173edd3d4 for child: 1405218340a9d9a499f287b55eca8ccc, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,717 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-79457093278d407b9ab5920b7232ec51 for child: 1405218340a9d9a499f287b55eca8ccc, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,718 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3452c0510f184df6bd9669c4f6c64cd0 for child: db1b569d476c9340ddb2430f438d297d, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,718 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3452c0510f184df6bd9669c4f6c64cd0 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,724 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1db77fb166b947fe82dd19088eda6129 2023-02-13 22:00:42,731 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/.splits/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-9cf556e5475141bbb6d23190acae7a51 for child: db1b569d476c9340ddb2430f438d297d, parent: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,731 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/9cf556e5475141bbb6d23190acae7a51 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,749 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/79457093278d407b9ab5920b7232ec51 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,749 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/bfc543b7ef0f40dd849ab48173edd3d4 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,756 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,775 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,775 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,778 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,783 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,791 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,799 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,801 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8 for region: b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:42,801 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region b54fd364e5081ef6e29844259f5ed76e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 22:00:42,839 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 22:00:42,841 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 22:00:42,842 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676325642842"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676325642842"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676325642842"}]},"ts":"1676325642842"} 2023-02-13 22:00:42,842 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676325642842"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325642842"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325642842"}]},"ts":"1676325642842"} 2023-02-13 22:00:42,842 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676325642842"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325642842"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325642842"}]},"ts":"1676325642842"} 2023-02-13 22:00:42,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1405218340a9d9a499f287b55eca8ccc, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db1b569d476c9340ddb2430f438d297d, ASSIGN}] 2023-02-13 22:00:42,846 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db1b569d476c9340ddb2430f438d297d, ASSIGN 2023-02-13 22:00:42,847 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db1b569d476c9340ddb2430f438d297d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,37979,1676325615388; forceNewPlan=false, retain=false 2023-02-13 22:00:42,848 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1405218340a9d9a499f287b55eca8ccc, ASSIGN 2023-02-13 22:00:42,848 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1405218340a9d9a499f287b55eca8ccc, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,37979,1676325615388; forceNewPlan=false, retain=false 2023-02-13 22:00:42,997 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 22:00:42,999 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=1405218340a9d9a499f287b55eca8ccc, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:42,999 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676325642999"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325642999"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325642999"}]},"ts":"1676325642999"} 2023-02-13 22:00:43,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 1405218340a9d9a499f287b55eca8ccc, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:00:43,006 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=db1b569d476c9340ddb2430f438d297d, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:43,006 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676325643006"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325643006"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325643006"}]},"ts":"1676325643006"} 2023-02-13 22:00:43,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure db1b569d476c9340ddb2430f438d297d, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:00:43,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1405218340a9d9a499f287b55eca8ccc, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.', STARTKEY => '', ENDKEY => '60'} 2023-02-13 22:00:43,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:43,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:43,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:43,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:43,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:43,167 INFO [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:43,168 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1 2023-02-13 22:00:43,168 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1 2023-02-13 22:00:43,168 INFO [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1405218340a9d9a499f287b55eca8ccc columnFamilyName cf1 2023-02-13 22:00:43,179 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,180 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129-bottom 2023-02-13 22:00:43,185 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b-bottom 2023-02-13 22:00:43,191 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8-bottom 2023-02-13 22:00:43,197 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603-bottom 2023-02-13 22:00:43,200 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3bb515fe234e430e8ba0e7e988e9d3c1 2023-02-13 22:00:43,204 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-4866a8208bd64630b1f438ea502970a4 2023-02-13 22:00:43,207 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-79457093278d407b9ab5920b7232ec51 2023-02-13 22:00:43,207 INFO [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(310): Store=1405218340a9d9a499f287b55eca8ccc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:43,207 INFO [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:43,208 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2 2023-02-13 22:00:43,208 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2 2023-02-13 22:00:43,208 INFO [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1405218340a9d9a499f287b55eca8ccc columnFamilyName cf2 2023-02-13 22:00:43,220 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae-bottom 2023-02-13 22:00:43,223 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,223 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52-bottom 2023-02-13 22:00:43,227 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f-bottom 2023-02-13 22:00:43,231 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169-bottom 2023-02-13 22:00:43,234 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-43964f3954e24bde8250ce9a01e9e21f 2023-02-13 22:00:43,237 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-8769cb8cd52441ddae4cd234c69dc2c9 2023-02-13 22:00:43,240 DEBUG [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-bfc543b7ef0f40dd849ab48173edd3d4 2023-02-13 22:00:43,240 INFO [StoreOpener-1405218340a9d9a499f287b55eca8ccc-1] regionserver.HStore(310): Store=1405218340a9d9a499f287b55eca8ccc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:43,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:43,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:43,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:43,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:43,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 1405218340a9d9a499f287b55eca8ccc; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66387287, jitterRate=-0.01075233519077301}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:43,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 1405218340a9d9a499f287b55eca8ccc: Running coprocessor pre-open hook at 1676325643163Writing region info on filesystem at 1676325643163Initializing all the Stores at 1676325643166 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325643166Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325643167 (+1 ms)Cleaning up temporary data from old regions at 1676325643242 (+75 ms)Cleaning up detritus from prior splits at 1676325643242Running coprocessor post-open hooks at 1676325643244 (+2 ms)Region opened successfully at 1676325643244 2023-02-13 22:00:43,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc., pid=77, masterSystemTime=1676325643154 2023-02-13 22:00:43,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:43,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:43,246 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:43,246 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:43,246 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,246 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,246 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1912): 1405218340a9d9a499f287b55eca8ccc/cf1 is initiating minor compaction (all files) 2023-02-13 22:00:43,246 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1912): 1405218340a9d9a499f287b55eca8ccc/cf2 is initiating minor compaction (all files) 2023-02-13 22:00:43,246 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1405218340a9d9a499f287b55eca8ccc/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,246 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1405218340a9d9a499f287b55eca8ccc/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,247 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-8769cb8cd52441ddae4cd234c69dc2c9, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-bfc543b7ef0f40dd849ab48173edd3d4, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-43964f3954e24bde8250ce9a01e9e21f, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52-bottom] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/.tmp, totalSize=35.0 K 2023-02-13 22:00:43,247 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-79457093278d407b9ab5920b7232ec51, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-4866a8208bd64630b1f438ea502970a4, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3bb515fe234e430e8ba0e7e988e9d3c1, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129-bottom] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/.tmp, totalSize=35.0 K 2023-02-13 22:00:43,247 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325640597 2023-02-13 22:00:43,247 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325640597 2023-02-13 22:00:43,247 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325640778 2023-02-13 22:00:43,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,247 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325640778 2023-02-13 22:00:43,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,248 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325640961 2023-02-13 22:00:43,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => db1b569d476c9340ddb2430f438d297d, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.', STARTKEY => '60', ENDKEY => ''} 2023-02-13 22:00:43,248 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=1405218340a9d9a499f287b55eca8ccc, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:43,248 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325640961 2023-02-13 22:00:43,248 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676325643248"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325643248"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325643248"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325643248"}]},"ts":"1676325643248"} 2023-02-13 22:00:43,248 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-79457093278d407b9ab5920b7232ec51, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325641161 2023-02-13 22:00:43,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:43,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:43,249 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-8769cb8cd52441ddae4cd234c69dc2c9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325641161 2023-02-13 22:00:43,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:43,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:43,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:43,249 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-bfc543b7ef0f40dd849ab48173edd3d4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325641343 2023-02-13 22:00:43,249 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-4866a8208bd64630b1f438ea502970a4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325641343 2023-02-13 22:00:43,250 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3bb515fe234e430e8ba0e7e988e9d3c1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325641535 2023-02-13 22:00:43,250 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-43964f3954e24bde8250ce9a01e9e21f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325641535 2023-02-13 22:00:43,251 INFO [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:43,251 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676325641706 2023-02-13 22:00:43,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-13 22:00:43,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 1405218340a9d9a499f287b55eca8ccc, server=jenkins-hbase20.apache.org,37979,1676325615388 in 249 msec 2023-02-13 22:00:43,252 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676325641706 2023-02-13 22:00:43,253 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1 2023-02-13 22:00:43,253 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1 2023-02-13 22:00:43,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1405218340a9d9a499f287b55eca8ccc, ASSIGN in 407 msec 2023-02-13 22:00:43,254 INFO [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db1b569d476c9340ddb2430f438d297d columnFamilyName cf1 2023-02-13 22:00:43,257 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 22:00:43,267 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1405218340a9d9a499f287b55eca8ccc#cf1#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:43,267 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] throttle.PressureAwareThroughputController(145): 1405218340a9d9a499f287b55eca8ccc#cf2#compaction#73 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:43,269 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,270 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129-top 2023-02-13 22:00:43,274 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b-top 2023-02-13 22:00:43,277 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8-top 2023-02-13 22:00:43,286 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603-top 2023-02-13 22:00:43,291 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3452c0510f184df6bd9669c4f6c64cd0 2023-02-13 22:00:43,292 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3452c0510f184df6bd9669c4f6c64cd0 2023-02-13 22:00:43,295 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9cf556e5475141bbb6d23190acae7a51 2023-02-13 22:00:43,295 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-9cf556e5475141bbb6d23190acae7a51 2023-02-13 22:00:43,296 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/.tmp/cf1/4b00e99b8ae94a8487827c9aa7a6f909 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/4b00e99b8ae94a8487827c9aa7a6f909 2023-02-13 22:00:43,298 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/.tmp/cf2/85baf68fc1244f7780c0ca4e3858957f as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/85baf68fc1244f7780c0ca4e3858957f 2023-02-13 22:00:43,299 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e3e91e846bf84ab0a0d6a5fc86ab4564 2023-02-13 22:00:43,299 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-e3e91e846bf84ab0a0d6a5fc86ab4564 2023-02-13 22:00:43,299 INFO [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(310): Store=db1b569d476c9340ddb2430f438d297d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:43,299 INFO [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:43,300 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2 2023-02-13 22:00:43,301 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2 2023-02-13 22:00:43,301 INFO [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db1b569d476c9340ddb2430f438d297d columnFamilyName cf2 2023-02-13 22:00:43,302 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1405218340a9d9a499f287b55eca8ccc/cf1 of 1405218340a9d9a499f287b55eca8ccc into 4b00e99b8ae94a8487827c9aa7a6f909(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:43,302 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1405218340a9d9a499f287b55eca8ccc: 2023-02-13 22:00:43,302 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc., storeName=1405218340a9d9a499f287b55eca8ccc/cf1, priority=9, startTime=1676325643245; duration=0sec 2023-02-13 22:00:43,302 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:43,302 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1405218340a9d9a499f287b55eca8ccc/cf2 of 1405218340a9d9a499f287b55eca8ccc into 85baf68fc1244f7780c0ca4e3858957f(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:43,303 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1405218340a9d9a499f287b55eca8ccc: 2023-02-13 22:00:43,303 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc., storeName=1405218340a9d9a499f287b55eca8ccc/cf2, priority=9, startTime=1676325643245; duration=0sec 2023-02-13 22:00:43,303 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:43,314 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae-top 2023-02-13 22:00:43,317 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,317 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52-top 2023-02-13 22:00:43,320 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f-top 2023-02-13 22:00:43,322 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169-top 2023-02-13 22:00:43,325 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1603140b7fe0439bbd846868340b5249 2023-02-13 22:00:43,325 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-1603140b7fe0439bbd846868340b5249 2023-02-13 22:00:43,329 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2aa0cf01906745a1be01e93962540832 2023-02-13 22:00:43,329 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-2aa0cf01906745a1be01e93962540832 2023-02-13 22:00:43,332 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f3476e1ed4c84940a753285a6304b428 2023-02-13 22:00:43,332 DEBUG [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-f3476e1ed4c84940a753285a6304b428 2023-02-13 22:00:43,332 INFO [StoreOpener-db1b569d476c9340ddb2430f438d297d-1] regionserver.HStore(310): Store=db1b569d476c9340ddb2430f438d297d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:43,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:43,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:43,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:00:43,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:43,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened db1b569d476c9340ddb2430f438d297d; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67037936, jitterRate=-0.0010569095611572266}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:00:43,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for db1b569d476c9340ddb2430f438d297d: Running coprocessor pre-open hook at 1676325643249Writing region info on filesystem at 1676325643249Initializing all the Stores at 1676325643250 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325643250Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325643251 (+1 ms)Cleaning up temporary data from old regions at 1676325643334 (+83 ms)Cleaning up detritus from prior splits at 1676325643335 (+1 ms)Running coprocessor post-open hooks at 1676325643336 (+1 ms)Region opened successfully at 1676325643336 2023-02-13 22:00:43,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d., pid=78, masterSystemTime=1676325643154 2023-02-13 22:00:43,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:43,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:43,338 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:43,338 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:00:43,338 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,338 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,338 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1912): db1b569d476c9340ddb2430f438d297d/cf1 is initiating minor compaction (all files) 2023-02-13 22:00:43,338 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1912): db1b569d476c9340ddb2430f438d297d/cf2 is initiating minor compaction (all files) 2023-02-13 22:00:43,338 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of db1b569d476c9340ddb2430f438d297d/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,338 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2258): Starting compaction of db1b569d476c9340ddb2430f438d297d/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,338 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-1603140b7fe0439bbd846868340b5249, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-2aa0cf01906745a1be01e93962540832, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-f3476e1ed4c84940a753285a6304b428] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/.tmp, totalSize=34.8 K 2023-02-13 22:00:43,338 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-e3e91e846bf84ab0a0d6a5fc86ab4564, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-9cf556e5475141bbb6d23190acae7a51, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3452c0510f184df6bd9669c4f6c64cd0] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/.tmp, totalSize=34.8 K 2023-02-13 22:00:43,339 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676325640597 2023-02-13 22:00:43,339 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676325640597 2023-02-13 22:00:43,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,339 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676325640778 2023-02-13 22:00:43,339 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676325640778 2023-02-13 22:00:43,340 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=db1b569d476c9340ddb2430f438d297d, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:43,340 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676325640961 2023-02-13 22:00:43,340 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676325640961 2023-02-13 22:00:43,340 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676325643339"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325643339"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325643339"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325643339"}]},"ts":"1676325643339"} 2023-02-13 22:00:43,340 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676325641706 2023-02-13 22:00:43,340 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676325641706 2023-02-13 22:00:43,340 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-1603140b7fe0439bbd846868340b5249, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 22:00:43,340 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-e3e91e846bf84ab0a0d6a5fc86ab4564, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 22:00:43,341 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-2aa0cf01906745a1be01e93962540832, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 22:00:43,341 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-9cf556e5475141bbb6d23190acae7a51, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 22:00:43,341 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-f3476e1ed4c84940a753285a6304b428, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 22:00:43,341 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3452c0510f184df6bd9669c4f6c64cd0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 22:00:43,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-13 22:00:43,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure db1b569d476c9340ddb2430f438d297d, server=jenkins-hbase20.apache.org,37979,1676325615388 in 332 msec 2023-02-13 22:00:43,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-13 22:00:43,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db1b569d476c9340ddb2430f438d297d, ASSIGN in 498 msec 2023-02-13 22:00:43,346 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d in 1.0420 sec 2023-02-13 22:00:43,358 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] throttle.PressureAwareThroughputController(145): db1b569d476c9340ddb2430f438d297d#cf1#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:43,358 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] throttle.PressureAwareThroughputController(145): db1b569d476c9340ddb2430f438d297d#cf2#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:43,375 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7c3178f7ae8f4ba0a39d63defc8a581e: NONE, but ROW specified in column family configuration 2023-02-13 22:00:43,375 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for a1c5ad3f4216406ba90285646e648a62: NONE, but ROW specified in column family configuration 2023-02-13 22:00:43,375 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/.tmp/cf1/7c3178f7ae8f4ba0a39d63defc8a581e as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/7c3178f7ae8f4ba0a39d63defc8a581e 2023-02-13 22:00:43,375 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/.tmp/cf2/a1c5ad3f4216406ba90285646e648a62 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/a1c5ad3f4216406ba90285646e648a62 2023-02-13 22:00:43,378 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7c3178f7ae8f4ba0a39d63defc8a581e: NONE, but ROW specified in column family configuration 2023-02-13 22:00:43,378 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for a1c5ad3f4216406ba90285646e648a62: NONE, but ROW specified in column family configuration 2023-02-13 22:00:43,379 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in db1b569d476c9340ddb2430f438d297d/cf2 of db1b569d476c9340ddb2430f438d297d into a1c5ad3f4216406ba90285646e648a62(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:43,379 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in db1b569d476c9340ddb2430f438d297d/cf1 of db1b569d476c9340ddb2430f438d297d into 7c3178f7ae8f4ba0a39d63defc8a581e(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:43,379 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for db1b569d476c9340ddb2430f438d297d: 2023-02-13 22:00:43,379 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for db1b569d476c9340ddb2430f438d297d: 2023-02-13 22:00:43,379 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d., storeName=db1b569d476c9340ddb2430f438d297d/cf2, priority=9, startTime=1676325643338; duration=0sec 2023-02-13 22:00:43,379 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d., storeName=db1b569d476c9340ddb2430f438d297d/cf1, priority=9, startTime=1676325643337; duration=0sec 2023-02-13 22:00:43,379 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:43,380 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:43,572 DEBUG [Listener at localhost.localdomain/42295] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 22:00:43,575 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:50950, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 22:00:43,576 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37979] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,577 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37979] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 22:00:43,577 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37979] regionserver.HStore(1912): 1405218340a9d9a499f287b55eca8ccc/cf1 is initiating major compaction (all files) 2023-02-13 22:00:43,577 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37979] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4acda858; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:43,577 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1405218340a9d9a499f287b55eca8ccc/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,577 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37979] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 22:00:43,577 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/4b00e99b8ae94a8487827c9aa7a6f909] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/.tmp, totalSize=6.8 K 2023-02-13 22:00:43,577 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37979] regionserver.HStore(1912): 1405218340a9d9a499f287b55eca8ccc/cf2 is initiating major compaction (all files) 2023-02-13 22:00:43,577 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37979] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@17a43ca2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:43,577 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1405218340a9d9a499f287b55eca8ccc/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,578 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/85baf68fc1244f7780c0ca4e3858957f] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/.tmp, totalSize=6.8 K 2023-02-13 22:00:43,578 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 4b00e99b8ae94a8487827c9aa7a6f909, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676325640597 2023-02-13 22:00:43,578 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 85baf68fc1244f7780c0ca4e3858957f, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676325640597 2023-02-13 22:00:43,580 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37979] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,580 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37979] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 22:00:43,580 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37979] regionserver.HStore(1912): db1b569d476c9340ddb2430f438d297d/cf1 is initiating major compaction (all files) 2023-02-13 22:00:43,580 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37979] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@282fd716; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:43,580 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37979] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 22:00:43,580 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37979] regionserver.HStore(1912): db1b569d476c9340ddb2430f438d297d/cf2 is initiating major compaction (all files) 2023-02-13 22:00:43,580 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37979] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@259e4605; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 22:00:43,581 INFO [Listener at localhost.localdomain/42295] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:43,583 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1405218340a9d9a499f287b55eca8ccc#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:43,585 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] throttle.PressureAwareThroughputController(145): 1405218340a9d9a499f287b55eca8ccc#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:43,597 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37979] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. is not mergeable because it has references 2023-02-13 22:00:43,597 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37979] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. is not mergeable because it has references 2023-02-13 22:00:43,600 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/.tmp/cf1/6e88e90e4e274fe9b7e55c480fd7896d as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/6e88e90e4e274fe9b7e55c480fd7896d 2023-02-13 22:00:43,603 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/.tmp/cf2/d1852bcbe1f545579b86e414b0a33909 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/d1852bcbe1f545579b86e414b0a33909 2023-02-13 22:00:43,604 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 1405218340a9d9a499f287b55eca8ccc/cf1 of 1405218340a9d9a499f287b55eca8ccc into 6e88e90e4e274fe9b7e55c480fd7896d(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:43,604 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1405218340a9d9a499f287b55eca8ccc: 2023-02-13 22:00:43,605 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676325643577; duration=0sec 2023-02-13 22:00:43,605 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 22:00:43,606 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of db1b569d476c9340ddb2430f438d297d/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,606 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/a1c5ad3f4216406ba90285646e648a62] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/.tmp, totalSize=5.0 K 2023-02-13 22:00:43,606 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting a1c5ad3f4216406ba90285646e648a62, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 22:00:43,608 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 1405218340a9d9a499f287b55eca8ccc/cf2 of 1405218340a9d9a499f287b55eca8ccc into d1852bcbe1f545579b86e414b0a33909(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:43,608 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1405218340a9d9a499f287b55eca8ccc: 2023-02-13 22:00:43,608 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676325643577; duration=0sec 2023-02-13 22:00:43,608 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:00:43,608 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2258): Starting compaction of db1b569d476c9340ddb2430f438d297d/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,608 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/7c3178f7ae8f4ba0a39d63defc8a581e] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/.tmp, totalSize=5.0 K 2023-02-13 22:00:43,608 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] throttle.PressureAwareThroughputController(145): db1b569d476c9340ddb2430f438d297d#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:43,609 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 7c3178f7ae8f4ba0a39d63defc8a581e, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 22:00:43,611 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] throttle.PressureAwareThroughputController(145): db1b569d476c9340ddb2430f438d297d#cf1#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:00:43,617 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 06b74f835e964631b6ebe39242822db2: NONE, but ROW specified in column family configuration 2023-02-13 22:00:43,618 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/.tmp/cf2/06b74f835e964631b6ebe39242822db2 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/06b74f835e964631b6ebe39242822db2 2023-02-13 22:00:43,621 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 06b74f835e964631b6ebe39242822db2: NONE, but ROW specified in column family configuration 2023-02-13 22:00:43,622 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 358ea838e9cc44788fc92ce4fd952263: NONE, but ROW specified in column family configuration 2023-02-13 22:00:43,622 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/.tmp/cf1/358ea838e9cc44788fc92ce4fd952263 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/358ea838e9cc44788fc92ce4fd952263 2023-02-13 22:00:43,622 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in db1b569d476c9340ddb2430f438d297d/cf2 of db1b569d476c9340ddb2430f438d297d into 06b74f835e964631b6ebe39242822db2(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:43,622 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for db1b569d476c9340ddb2430f438d297d: 2023-02-13 22:00:43,623 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676325643580; duration=0sec 2023-02-13 22:00:43,623 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:43,626 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 358ea838e9cc44788fc92ce4fd952263: NONE, but ROW specified in column family configuration 2023-02-13 22:00:43,626 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in db1b569d476c9340ddb2430f438d297d/cf1 of db1b569d476c9340ddb2430f438d297d into 358ea838e9cc44788fc92ce4fd952263(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:00:43,626 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for db1b569d476c9340ddb2430f438d297d: 2023-02-13 22:00:43,627 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676325643580; duration=0sec 2023-02-13 22:00:43,627 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:00:43,711 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37979] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. is not mergeable because it has references 2023-02-13 22:00:43,712 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37979] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. is not mergeable because it has references 2023-02-13 22:00:43,718 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-13 22:00:43,719 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:43,720 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:43,721 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:43,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:43,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 22:00:43,724 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325643724"}]},"ts":"1676325643724"} 2023-02-13 22:00:43,725 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-13 22:00:43,726 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-13 22:00:43,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1405218340a9d9a499f287b55eca8ccc, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db1b569d476c9340ddb2430f438d297d, UNASSIGN}] 2023-02-13 22:00:43,728 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db1b569d476c9340ddb2430f438d297d, UNASSIGN 2023-02-13 22:00:43,729 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=db1b569d476c9340ddb2430f438d297d, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:43,729 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676325643729"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325643729"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325643729"}]},"ts":"1676325643729"} 2023-02-13 22:00:43,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure db1b569d476c9340ddb2430f438d297d, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:00:43,732 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1405218340a9d9a499f287b55eca8ccc, UNASSIGN 2023-02-13 22:00:43,733 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=1405218340a9d9a499f287b55eca8ccc, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:43,733 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676325643733"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325643733"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325643733"}]},"ts":"1676325643733"} 2023-02-13 22:00:43,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 1405218340a9d9a499f287b55eca8ccc, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:00:43,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 22:00:43,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:43,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing db1b569d476c9340ddb2430f438d297d, disabling compactions & flushes 2023-02-13 22:00:43,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. after waiting 0 ms 2023-02-13 22:00:43,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,885 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-e3e91e846bf84ab0a0d6a5fc86ab4564, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-9cf556e5475141bbb6d23190acae7a51, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/7c3178f7ae8f4ba0a39d63defc8a581e, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3452c0510f184df6bd9669c4f6c64cd0] to archive 2023-02-13 22:00:43,887 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:43,892 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,895 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,897 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,899 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,902 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-e3e91e846bf84ab0a0d6a5fc86ab4564 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-e3e91e846bf84ab0a0d6a5fc86ab4564 2023-02-13 22:00:43,904 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-9cf556e5475141bbb6d23190acae7a51 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-9cf556e5475141bbb6d23190acae7a51 2023-02-13 22:00:43,905 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/7c3178f7ae8f4ba0a39d63defc8a581e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/7c3178f7ae8f4ba0a39d63defc8a581e 2023-02-13 22:00:43,907 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3452c0510f184df6bd9669c4f6c64cd0 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3452c0510f184df6bd9669c4f6c64cd0 2023-02-13 22:00:43,909 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-1603140b7fe0439bbd846868340b5249, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-2aa0cf01906745a1be01e93962540832, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/a1c5ad3f4216406ba90285646e648a62, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-f3476e1ed4c84940a753285a6304b428] to archive 2023-02-13 22:00:43,910 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:43,912 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,913 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,915 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,916 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,918 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-1603140b7fe0439bbd846868340b5249 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-1603140b7fe0439bbd846868340b5249 2023-02-13 22:00:43,919 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-2aa0cf01906745a1be01e93962540832 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-2aa0cf01906745a1be01e93962540832 2023-02-13 22:00:43,921 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/a1c5ad3f4216406ba90285646e648a62 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/a1c5ad3f4216406ba90285646e648a62 2023-02-13 22:00:43,922 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-f3476e1ed4c84940a753285a6304b428 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-f3476e1ed4c84940a753285a6304b428 2023-02-13 22:00:43,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 22:00:43,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:43,928 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d. 2023-02-13 22:00:43,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for db1b569d476c9340ddb2430f438d297d: Waiting for close lock at 1676325643883Running coprocessor pre-close hooks at 1676325643883Disabling compacts and flushes for region at 1676325643883Disabling writes for close at 1676325643883Writing region close event to WAL at 1676325643923 (+40 ms)Running coprocessor post-close hooks at 1676325643927 (+4 ms)Closed at 1676325643928 (+1 ms) 2023-02-13 22:00:43,929 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:43,929 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:43,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 1405218340a9d9a499f287b55eca8ccc, disabling compactions & flushes 2023-02-13 22:00:43,929 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. after waiting 0 ms 2023-02-13 22:00:43,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,930 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=db1b569d476c9340ddb2430f438d297d, regionState=CLOSED 2023-02-13 22:00:43,930 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676325643929"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325643929"}]},"ts":"1676325643929"} 2023-02-13 22:00:43,930 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-79457093278d407b9ab5920b7232ec51, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-4866a8208bd64630b1f438ea502970a4, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3bb515fe234e430e8ba0e7e988e9d3c1, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/4b00e99b8ae94a8487827c9aa7a6f909, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129-bottom] to archive 2023-02-13 22:00:43,931 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:43,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-13 22:00:43,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure db1b569d476c9340ddb2430f438d297d, server=jenkins-hbase20.apache.org,37979,1676325615388 in 201 msec 2023-02-13 22:00:43,933 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/86e3cd83867740c3801f67d200ecc603.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db1b569d476c9340ddb2430f438d297d, UNASSIGN in 205 msec 2023-02-13 22:00:43,934 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/803226c1943e4bb69ca383613542d1b8.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,934 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/58a957378f384351a76d1eafde7ebe1b.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,935 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-79457093278d407b9ab5920b7232ec51 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-79457093278d407b9ab5920b7232ec51 2023-02-13 22:00:43,936 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-4866a8208bd64630b1f438ea502970a4 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-4866a8208bd64630b1f438ea502970a4 2023-02-13 22:00:43,937 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3bb515fe234e430e8ba0e7e988e9d3c1 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-3bb515fe234e430e8ba0e7e988e9d3c1 2023-02-13 22:00:43,938 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/4b00e99b8ae94a8487827c9aa7a6f909 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/4b00e99b8ae94a8487827c9aa7a6f909 2023-02-13 22:00:43,939 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/1db77fb166b947fe82dd19088eda6129.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,941 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-8769cb8cd52441ddae4cd234c69dc2c9, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-bfc543b7ef0f40dd849ab48173edd3d4, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-43964f3954e24bde8250ce9a01e9e21f, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/85baf68fc1244f7780c0ca4e3858957f, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52-bottom] to archive 2023-02-13 22:00:43,942 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:00:43,943 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/283756b61cff410eb263f79d83555dae.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,944 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/ae159d6b94a0429baa84dd1235786f2f.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,945 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/c2884f31c5ef45a895c688ccc7bbd169.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,946 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-8769cb8cd52441ddae4cd234c69dc2c9 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-8769cb8cd52441ddae4cd234c69dc2c9 2023-02-13 22:00:43,946 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-bfc543b7ef0f40dd849ab48173edd3d4 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-bfc543b7ef0f40dd849ab48173edd3d4 2023-02-13 22:00:43,947 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-43964f3954e24bde8250ce9a01e9e21f to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/testSplitTableRegionDeletedRowsDaughter=b54fd364e5081ef6e29844259f5ed76e-43964f3954e24bde8250ce9a01e9e21f 2023-02-13 22:00:43,948 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/85baf68fc1244f7780c0ca4e3858957f to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/85baf68fc1244f7780c0ca4e3858957f 2023-02-13 22:00:43,949 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/7ae1fa8f0b29466988677342f52e7c52.b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:43,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 22:00:43,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:43,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc. 2023-02-13 22:00:43,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 1405218340a9d9a499f287b55eca8ccc: Waiting for close lock at 1676325643929Running coprocessor pre-close hooks at 1676325643929Disabling compacts and flushes for region at 1676325643929Disabling writes for close at 1676325643929Writing region close event to WAL at 1676325643950 (+21 ms)Running coprocessor post-close hooks at 1676325643952 (+2 ms)Closed at 1676325643953 (+1 ms) 2023-02-13 22:00:43,954 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:43,954 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=1405218340a9d9a499f287b55eca8ccc, regionState=CLOSED 2023-02-13 22:00:43,954 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676325643954"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325643954"}]},"ts":"1676325643954"} 2023-02-13 22:00:43,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-13 22:00:43,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 1405218340a9d9a499f287b55eca8ccc, server=jenkins-hbase20.apache.org,37979,1676325615388 in 221 msec 2023-02-13 22:00:43,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-13 22:00:43,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1405218340a9d9a499f287b55eca8ccc, UNASSIGN in 230 msec 2023-02-13 22:00:43,959 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325643959"}]},"ts":"1676325643959"} 2023-02-13 22:00:43,960 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-13 22:00:43,961 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-13 22:00:43,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 241 msec 2023-02-13 22:00:44,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 22:00:44,027 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-13 22:00:44,028 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:44,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:44,031 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:44,032 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:44,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 22:00:44,037 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:44,037 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:44,037 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:44,040 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/recovered.edits] 2023-02-13 22:00:44,040 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/recovered.edits] 2023-02-13 22:00:44,040 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/recovered.edits] 2023-02-13 22:00:44,044 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/358ea838e9cc44788fc92ce4fd952263 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf1/358ea838e9cc44788fc92ce4fd952263 2023-02-13 22:00:44,044 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/1db77fb166b947fe82dd19088eda6129 2023-02-13 22:00:44,045 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/6e88e90e4e274fe9b7e55c480fd7896d to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf1/6e88e90e4e274fe9b7e55c480fd7896d 2023-02-13 22:00:44,046 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3452c0510f184df6bd9669c4f6c64cd0 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3452c0510f184df6bd9669c4f6c64cd0 2023-02-13 22:00:44,051 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/d1852bcbe1f545579b86e414b0a33909 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/cf2/d1852bcbe1f545579b86e414b0a33909 2023-02-13 22:00:44,051 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/06b74f835e964631b6ebe39242822db2 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/cf2/06b74f835e964631b6ebe39242822db2 2023-02-13 22:00:44,051 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3bb515fe234e430e8ba0e7e988e9d3c1 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/3bb515fe234e430e8ba0e7e988e9d3c1 2023-02-13 22:00:44,052 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/4866a8208bd64630b1f438ea502970a4 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/4866a8208bd64630b1f438ea502970a4 2023-02-13 22:00:44,054 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/58a957378f384351a76d1eafde7ebe1b 2023-02-13 22:00:44,054 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/recovered.edits/113.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc/recovered.edits/113.seqid 2023-02-13 22:00:44,054 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/recovered.edits/113.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d/recovered.edits/113.seqid 2023-02-13 22:00:44,054 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1405218340a9d9a499f287b55eca8ccc 2023-02-13 22:00:44,054 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db1b569d476c9340ddb2430f438d297d 2023-02-13 22:00:44,055 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/79457093278d407b9ab5920b7232ec51 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/79457093278d407b9ab5920b7232ec51 2023-02-13 22:00:44,056 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/803226c1943e4bb69ca383613542d1b8 2023-02-13 22:00:44,057 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/86e3cd83867740c3801f67d200ecc603 2023-02-13 22:00:44,058 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/9cf556e5475141bbb6d23190acae7a51 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/9cf556e5475141bbb6d23190acae7a51 2023-02-13 22:00:44,058 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/e3e91e846bf84ab0a0d6a5fc86ab4564 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf1/e3e91e846bf84ab0a0d6a5fc86ab4564 2023-02-13 22:00:44,060 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/1603140b7fe0439bbd846868340b5249 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/1603140b7fe0439bbd846868340b5249 2023-02-13 22:00:44,061 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/283756b61cff410eb263f79d83555dae 2023-02-13 22:00:44,062 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/2aa0cf01906745a1be01e93962540832 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/2aa0cf01906745a1be01e93962540832 2023-02-13 22:00:44,063 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/43964f3954e24bde8250ce9a01e9e21f to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/43964f3954e24bde8250ce9a01e9e21f 2023-02-13 22:00:44,064 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/7ae1fa8f0b29466988677342f52e7c52 2023-02-13 22:00:44,065 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/8769cb8cd52441ddae4cd234c69dc2c9 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/8769cb8cd52441ddae4cd234c69dc2c9 2023-02-13 22:00:44,066 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/ae159d6b94a0429baa84dd1235786f2f 2023-02-13 22:00:44,066 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/bfc543b7ef0f40dd849ab48173edd3d4 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/bfc543b7ef0f40dd849ab48173edd3d4 2023-02-13 22:00:44,067 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/c2884f31c5ef45a895c688ccc7bbd169 2023-02-13 22:00:44,068 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/f3476e1ed4c84940a753285a6304b428 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/cf2/f3476e1ed4c84940a753285a6304b428 2023-02-13 22:00:44,070 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/recovered.edits/106.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e/recovered.edits/106.seqid 2023-02-13 22:00:44,070 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b54fd364e5081ef6e29844259f5ed76e 2023-02-13 22:00:44,070 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 22:00:44,072 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:44,074 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-13 22:00:44,075 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-13 22:00:44,077 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:44,077 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-13 22:00:44,077 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325644077"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:44,077 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325644077"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:44,077 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325644077"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:44,079 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 22:00:44,079 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b54fd364e5081ef6e29844259f5ed76e, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676325640089.b54fd364e5081ef6e29844259f5ed76e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 1405218340a9d9a499f287b55eca8ccc, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676325642303.1405218340a9d9a499f287b55eca8ccc.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => db1b569d476c9340ddb2430f438d297d, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676325642303.db1b569d476c9340ddb2430f438d297d.', STARTKEY => '60', ENDKEY => ''}] 2023-02-13 22:00:44,080 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-13 22:00:44,080 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676325644080"}]},"ts":"9223372036854775807"} 2023-02-13 22:00:44,081 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-13 22:00:44,083 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:00:44,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 55 msec 2023-02-13 22:00:44,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 22:00:44,134 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-13 22:00:44,155 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=411 (was 400) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:43420 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:34182 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:41050 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:43300 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:54344 [Waiting for operation #43] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:55450 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:43226 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:43352 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:43768 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:43704 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase20:37979-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:54402 [Waiting for operation #49] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:41046 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:57480 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e26826a-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:55428 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:40664 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_867542362_15 at /127.0.0.1:41138 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=852 (was 850) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=269 (was 269), ProcessCount=162 (was 162), AvailableMemoryMB=7637 (was 7703) 2023-02-13 22:00:44,170 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=411, OpenFileDescriptor=852, MaxFileDescriptor=60000, SystemLoadAverage=269, ProcessCount=162, AvailableMemoryMB=7636 2023-02-13 22:00:44,170 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:00:44,173 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:00:44,173 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-13 22:00:44,174 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:00:44,176 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:00:44,176 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:00:44,178 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,178 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9 empty. 2023-02-13 22:00:44,178 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,178 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 22:00:44,190 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:44,191 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => faa5a059f8b3120d55e00b3ba9ebb6b9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:00:44,199 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:44,199 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing faa5a059f8b3120d55e00b3ba9ebb6b9, disabling compactions & flushes 2023-02-13 22:00:44,199 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,199 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,199 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. after waiting 0 ms 2023-02-13 22:00:44,199 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,199 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,199 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for faa5a059f8b3120d55e00b3ba9ebb6b9: Waiting for close lock at 1676325644199Disabling compacts and flushes for region at 1676325644199Disabling writes for close at 1676325644199Writing region close event to WAL at 1676325644199Closed at 1676325644199 2023-02-13 22:00:44,201 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:00:44,202 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325644202"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325644202"}]},"ts":"1676325644202"} 2023-02-13 22:00:44,203 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:00:44,203 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:00:44,204 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325644204"}]},"ts":"1676325644204"} 2023-02-13 22:00:44,204 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-13 22:00:44,208 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:44,209 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:44,209 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:44,209 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:44,209 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:44,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, ASSIGN}] 2023-02-13 22:00:44,211 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, ASSIGN 2023-02-13 22:00:44,211 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37979,1676325615388; forceNewPlan=false, retain=false 2023-02-13 22:00:44,362 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:44,366 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=faa5a059f8b3120d55e00b3ba9ebb6b9, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:44,367 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325644366"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325644366"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325644366"}]},"ts":"1676325644366"} 2023-02-13 22:00:44,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure faa5a059f8b3120d55e00b3ba9ebb6b9, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:00:44,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => faa5a059f8b3120d55e00b3ba9ebb6b9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:44,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:44,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:44,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,535 INFO [StoreOpener-faa5a059f8b3120d55e00b3ba9ebb6b9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,536 DEBUG [StoreOpener-faa5a059f8b3120d55e00b3ba9ebb6b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/cf1 2023-02-13 22:00:44,537 DEBUG [StoreOpener-faa5a059f8b3120d55e00b3ba9ebb6b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/cf1 2023-02-13 22:00:44,537 INFO [StoreOpener-faa5a059f8b3120d55e00b3ba9ebb6b9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region faa5a059f8b3120d55e00b3ba9ebb6b9 columnFamilyName cf1 2023-02-13 22:00:44,538 INFO [StoreOpener-faa5a059f8b3120d55e00b3ba9ebb6b9-1] regionserver.HStore(310): Store=faa5a059f8b3120d55e00b3ba9ebb6b9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:44,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:44,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened faa5a059f8b3120d55e00b3ba9ebb6b9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61772288, jitterRate=-0.07952117919921875}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:44,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for faa5a059f8b3120d55e00b3ba9ebb6b9: Running coprocessor pre-open hook at 1676325644532Writing region info on filesystem at 1676325644532Initializing all the Stores at 1676325644534 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325644534Cleaning up temporary data from old regions at 1676325644540 (+6 ms)Cleaning up detritus from prior splits at 1676325644540Running coprocessor post-open hooks at 1676325644545 (+5 ms)Region opened successfully at 1676325644545 2023-02-13 22:00:44,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9., pid=87, masterSystemTime=1676325644523 2023-02-13 22:00:44,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=faa5a059f8b3120d55e00b3ba9ebb6b9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:44,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325644548"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325644548"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325644548"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325644548"}]},"ts":"1676325644548"} 2023-02-13 22:00:44,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-13 22:00:44,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure faa5a059f8b3120d55e00b3ba9ebb6b9, server=jenkins-hbase20.apache.org,37979,1676325615388 in 179 msec 2023-02-13 22:00:44,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-13 22:00:44,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, ASSIGN in 341 msec 2023-02-13 22:00:44,551 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:00:44,551 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325644551"}]},"ts":"1676325644551"} 2023-02-13 22:00:44,552 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-13 22:00:44,553 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:00:44,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 381 msec 2023-02-13 22:00:44,686 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$12(2441): Client=jenkins//148.251.75.209 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 22:00:44,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:00:44,705 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 22:00:44,706 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:44,706 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 22:00:44,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-13 22:00:44,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, REOPEN/MOVE}] 2023-02-13 22:00:44,722 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, REOPEN/MOVE 2023-02-13 22:00:44,723 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=faa5a059f8b3120d55e00b3ba9ebb6b9, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:44,723 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325644723"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325644723"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325644723"}]},"ts":"1676325644723"} 2023-02-13 22:00:44,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure faa5a059f8b3120d55e00b3ba9ebb6b9, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:00:44,877 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing faa5a059f8b3120d55e00b3ba9ebb6b9, disabling compactions & flushes 2023-02-13 22:00:44,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. after waiting 0 ms 2023-02-13 22:00:44,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 22:00:44,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:00:44,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:44,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for faa5a059f8b3120d55e00b3ba9ebb6b9: Waiting for close lock at 1676325644877Running coprocessor pre-close hooks at 1676325644877Disabling compacts and flushes for region at 1676325644877Disabling writes for close at 1676325644879 (+2 ms)Writing region close event to WAL at 1676325644880 (+1 ms)Running coprocessor post-close hooks at 1676325644889 (+9 ms)Closed at 1676325644889 2023-02-13 22:00:44,889 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(3509): Not adding moved region record: faa5a059f8b3120d55e00b3ba9ebb6b9 to self. 2023-02-13 22:00:44,892 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:44,892 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=faa5a059f8b3120d55e00b3ba9ebb6b9, regionState=CLOSED 2023-02-13 22:00:44,892 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325644892"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325644892"}]},"ts":"1676325644892"} 2023-02-13 22:00:44,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-13 22:00:44,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure faa5a059f8b3120d55e00b3ba9ebb6b9, server=jenkins-hbase20.apache.org,37979,1676325615388 in 170 msec 2023-02-13 22:00:44,896 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase20.apache.org,37979,1676325615388; forceNewPlan=false, retain=true 2023-02-13 22:00:45,046 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:45,046 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=faa5a059f8b3120d55e00b3ba9ebb6b9, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:45,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325645046"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325645046"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325645046"}]},"ts":"1676325645046"} 2023-02-13 22:00:45,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure faa5a059f8b3120d55e00b3ba9ebb6b9, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:00:45,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:45,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => faa5a059f8b3120d55e00b3ba9ebb6b9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:45,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:45,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:45,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:45,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:45,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:45,207 INFO [StoreOpener-faa5a059f8b3120d55e00b3ba9ebb6b9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:45,209 DEBUG [StoreOpener-faa5a059f8b3120d55e00b3ba9ebb6b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/cf1 2023-02-13 22:00:45,209 DEBUG [StoreOpener-faa5a059f8b3120d55e00b3ba9ebb6b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/cf1 2023-02-13 22:00:45,210 INFO [StoreOpener-faa5a059f8b3120d55e00b3ba9ebb6b9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region faa5a059f8b3120d55e00b3ba9ebb6b9 columnFamilyName cf1 2023-02-13 22:00:45,211 INFO [StoreOpener-faa5a059f8b3120d55e00b3ba9ebb6b9-1] regionserver.HStore(310): Store=faa5a059f8b3120d55e00b3ba9ebb6b9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:45,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:45,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:45,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:00:45,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened faa5a059f8b3120d55e00b3ba9ebb6b9; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64656227, jitterRate=-0.036547139286994934}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:45,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for faa5a059f8b3120d55e00b3ba9ebb6b9: Running coprocessor pre-open hook at 1676325645205Writing region info on filesystem at 1676325645205Initializing all the Stores at 1676325645207 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325645207Cleaning up temporary data from old regions at 1676325645216 (+9 ms)Cleaning up detritus from prior splits at 1676325645216Running coprocessor post-open hooks at 1676325645222 (+6 ms)Region opened successfully at 1676325645222 2023-02-13 22:00:45,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9., pid=92, masterSystemTime=1676325645200 2023-02-13 22:00:45,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:45,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:00:45,225 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=faa5a059f8b3120d55e00b3ba9ebb6b9, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:00:45,225 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325645225"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325645225"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325645225"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325645225"}]},"ts":"1676325645225"} 2023-02-13 22:00:45,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-13 22:00:45,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure faa5a059f8b3120d55e00b3ba9ebb6b9, server=jenkins-hbase20.apache.org,37979,1676325615388 in 179 msec 2023-02-13 22:00:45,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-13 22:00:45,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, REOPEN/MOVE in 508 msec 2023-02-13 22:00:45,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-13 22:00:45,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 525 msec 2023-02-13 22:00:45,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3, ASSIGN}] 2023-02-13 22:00:45,239 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3, ASSIGN 2023-02-13 22:00:45,240 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 22:00:45,391 DEBUG [jenkins-hbase20:36939] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:45,393 DEBUG [jenkins-hbase20:36939] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:45,393 DEBUG [jenkins-hbase20:36939] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:45,393 DEBUG [jenkins-hbase20:36939] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:45,393 DEBUG [jenkins-hbase20:36939] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-13 22:00:45,396 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=08a32e58e596f7f48d8374c3c2c4f4b3, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:00:45,396 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325645396"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1676325645396"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676325645396"}]},"ts":"1676325645396"} 2023-02-13 22:00:45,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 08a32e58e596f7f48d8374c3c2c4f4b3, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:00:45,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3. 2023-02-13 22:00:45,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 08a32e58e596f7f48d8374c3c2c4f4b3, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-13 22:00:45,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:00:45,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 08a32e58e596f7f48d8374c3c2c4f4b3 2023-02-13 22:00:45,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:45,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 08a32e58e596f7f48d8374c3c2c4f4b3 2023-02-13 22:00:45,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 08a32e58e596f7f48d8374c3c2c4f4b3 2023-02-13 22:00:45,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:45,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:45,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:45,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:45,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:46,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:46,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:46,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:46,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:46,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:46,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:46,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:46,744 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-13 22:00:46,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:46,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:46,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:47,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:47,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:47,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:47,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:47,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:47,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:47,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:47,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:47,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:47,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:48,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:48,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:48,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:48,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:48,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:48,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:48,667 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 22:00:48,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:48,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:48,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:48,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:49,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:49,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:49,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:49,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:49,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:49,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:49,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:49,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:49,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:49,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:50,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:50,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:50,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:50,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:50,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:50,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:50,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:50,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:50,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:50,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:51,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:51,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:51,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:51,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:51,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:51,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:51,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:51,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:51,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:51,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:52,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:52,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:52,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:52,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:52,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:52,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:52,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:52,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:52,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:52,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:53,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:53,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:53,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:53,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:53,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:53,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:53,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:53,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:53,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:53,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:54,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:54,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:54,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:54,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:54,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:54,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:54,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:54,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:54,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:55,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:55,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:55,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:55,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:55,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:55,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:55,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:55,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:55,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:55,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:56,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:56,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:56,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:56,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:56,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:56,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:56,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:56,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:56,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:56,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:57,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:57,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:57,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:57,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:57,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:57,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:57,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:57,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:57,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:57,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:58,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:58,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:58,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:58,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:58,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:58,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:58,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:58,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:58,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:58,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:59,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:59,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:59,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:59,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:59,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:59,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:59,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:59,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:59,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:00:59,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:00,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:00,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:00,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:00,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:00,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:00,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:00,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:00,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:00,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:00,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:01,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:01,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:01,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:01,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:01,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:01,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:01,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:01,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:01,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:01,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:02,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:02,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:02,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:02,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:02,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:02,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:02,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:02,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:02,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:02,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:03,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:03,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:03,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:03,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:03,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:03,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:03,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:03,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:03,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:03,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:04,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:04,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:04,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:04,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:04,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:04,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:04,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:04,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:04,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:04,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:05,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:05,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:05,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:05,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:05,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:05,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:05,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:05,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:05,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:05,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:06,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:06,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:06,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:06,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:06,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:06,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:06,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:06,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:06,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:06,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:07,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:07,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:07,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:07,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:07,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:07,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:07,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:07,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:07,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:07,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:08,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:08,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:08,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:08,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:08,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:08,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:08,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:08,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:08,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:08,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:09,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:09,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:09,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:09,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:09,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:09,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:09,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:09,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:09,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:09,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:10,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:10,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:10,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:10,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:10,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:10,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:10,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:10,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:10,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:10,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:11,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:11,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:11,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:11,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:11,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:11,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:11,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:11,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:11,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:11,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:12,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:12,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:12,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:12,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:12,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:12,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:12,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:12,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:12,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:12,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:13,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:13,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:13,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:13,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:13,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:13,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:13,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:13,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:13,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:13,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:14,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:14,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:14,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:14,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:14,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:14,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:14,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:14,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:14,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:14,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:15,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:15,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:15,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:15,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:15,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:15,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:15,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:15,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:15,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:15,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:16,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:16,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:16,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:16,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:16,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:16,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:16,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:16,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:16,780 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 22:01:16,804 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 22:01:16,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:16,871 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 22:01:16,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:17,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:17,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:17,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:17,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:17,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:17,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:17,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:17,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:17,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:17,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:18,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:18,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:18,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:18,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:18,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:18,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:18,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:18,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:18,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:18,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:19,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:19,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:19,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:19,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:19,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:19,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:19,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:19,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:19,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:19,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:20,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:20,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:20,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:20,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:20,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:20,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:20,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:20,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:20,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:20,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:21,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:21,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:21,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:21,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:21,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:21,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:21,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:21,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:21,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:21,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:22,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:22,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:22,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:22,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:22,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:22,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:22,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:22,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:22,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:22,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:23,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:23,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:23,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:23,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:23,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:23,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:23,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:23,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:23,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:23,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:24,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:24,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:24,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:24,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:24,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:24,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:24,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:24,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:24,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:24,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:25,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:25,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:25,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:25,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:25,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:25,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:25,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:25,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:25,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:25,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:26,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:26,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:26,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:26,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:26,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:26,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:26,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:26,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:26,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:26,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:27,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:27,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:27,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:27,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:27,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:27,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:27,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:27,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:27,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:28,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:28,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:28,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:28,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:28,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:28,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:28,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:28,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:28,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:28,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:29,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:29,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:29,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:29,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:29,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:29,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:29,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:29,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:29,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:29,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:30,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:30,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:30,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:30,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:30,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:30,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:30,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:30,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:30,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:30,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:31,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:31,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:31,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:31,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:31,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:31,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:31,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:31,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:31,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:31,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:32,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:32,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:32,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:32,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:32,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:32,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:32,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:32,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:32,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:32,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:33,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:33,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:33,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:33,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:33,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:33,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:33,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:33,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:33,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:33,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:34,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:34,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:34,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:34,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:34,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:34,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:34,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:34,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:34,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:34,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:35,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:35,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:35,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:35,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:35,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:35,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:35,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:35,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:35,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:35,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:36,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:36,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:36,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:36,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:36,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:36,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:36,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:36,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:36,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:36,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:37,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:37,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:37,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:37,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:37,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:37,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:37,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:37,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:37,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:37,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:38,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:38,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:38,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:38,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:38,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:38,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:38,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:38,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:38,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:38,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:39,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:39,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:39,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:39,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:39,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:39,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:39,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:39,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:39,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:39,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:40,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:40,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:40,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:40,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:40,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:40,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:40,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:40,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:40,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:40,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:41,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:41,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:41,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:41,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:41,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:41,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:41,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:41,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:41,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:41,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:42,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:42,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:42,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:42,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:42,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:42,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:42,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:42,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:42,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:42,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:43,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:43,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:43,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:43,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:43,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:43,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:43,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:43,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:43,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:43,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:44,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:44,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:44,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:44,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:44,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:44,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:44,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:44,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:44,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 22:01:44,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:44,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 22:01:44,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:45,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:45,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 22:01:45,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:45,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:45,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:45,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 22:01:45,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:45,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:45,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:45,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:45,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:45,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 22:01:45,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:46,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:46,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:46,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:46,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:46,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:46,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:46,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:46,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:46,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:46,923 DEBUG [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:46,925 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:46,926 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:46,926 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:46,927 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:46,930 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting faa5a059f8b3120d55e00b3ba9ebb6b9, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:46,935 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 exec-time=11 msec 2023-02-13 22:01:46,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:47,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:47,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 22:01:47,260 INFO [StoreOpener-08a32e58e596f7f48d8374c3c2c4f4b3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 08a32e58e596f7f48d8374c3c2c4f4b3 2023-02-13 22:01:47,265 DEBUG [StoreOpener-08a32e58e596f7f48d8374c3c2c4f4b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/cf1 2023-02-13 22:01:47,265 DEBUG [StoreOpener-08a32e58e596f7f48d8374c3c2c4f4b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/cf1 2023-02-13 22:01:47,266 INFO [StoreOpener-08a32e58e596f7f48d8374c3c2c4f4b3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08a32e58e596f7f48d8374c3c2c4f4b3 columnFamilyName cf1 2023-02-13 22:01:47,268 INFO [StoreOpener-08a32e58e596f7f48d8374c3c2c4f4b3-1] regionserver.HStore(310): Store=08a32e58e596f7f48d8374c3c2c4f4b3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:47,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 08a32e58e596f7f48d8374c3c2c4f4b3; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73097289, jitterRate=0.08923448622226715}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:01:47,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 08a32e58e596f7f48d8374c3c2c4f4b3: Running coprocessor pre-open hook at 1676325645573Initializing all the Stores at 1676325707259 (+61686 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325707259Running coprocessor post-open hooks at 1676325707272 (+13 ms)Region opened successfully at 1676325707272 2023-02-13 22:01:47,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3., pid=94, masterSystemTime=1676325645557 2023-02-13 22:01:47,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3. 2023-02-13 22:01:47,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3. 2023-02-13 22:01:47,276 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=08a32e58e596f7f48d8374c3c2c4f4b3, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:47,276 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325707275"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1676325707275"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676325707275"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676325707275"}]},"ts":"1676325707275"} 2023-02-13 22:01:47,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-13 22:01:47,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 08a32e58e596f7f48d8374c3c2c4f4b3, server=jenkins-hbase20.apache.org,42047,1676325615467 in 1 mins, 1.877 sec 2023-02-13 22:01:47,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-13 22:01:47,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3, ASSIGN in 1 mins, 2.042 sec 2023-02-13 22:01:47,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.594 sec 2023-02-13 22:01:47,285 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676325644683","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62602,"client":"148.251.75.209:54362","queuetimems":0,"class":"HMaster"} 2023-02-13 22:01:47,286 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-13 22:01:49,176 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up 2023-02-13 22:01:49,180 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:01:49,183 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:49,184 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:49,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:49,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 22:01:49,189 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325709189"}]},"ts":"1676325709189"} 2023-02-13 22:01:49,190 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-13 22:01:49,192 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-13 22:01:49,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3, UNASSIGN}] 2023-02-13 22:01:49,193 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3, UNASSIGN 2023-02-13 22:01:49,194 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=08a32e58e596f7f48d8374c3c2c4f4b3, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:49,194 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325709194"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1676325709194"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676325709194"}]},"ts":"1676325709194"} 2023-02-13 22:01:49,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 08a32e58e596f7f48d8374c3c2c4f4b3, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:01:49,198 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, UNASSIGN 2023-02-13 22:01:49,199 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=faa5a059f8b3120d55e00b3ba9ebb6b9, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:49,199 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325709199"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325709199"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325709199"}]},"ts":"1676325709199"} 2023-02-13 22:01:49,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure faa5a059f8b3120d55e00b3ba9ebb6b9, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:01:49,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 22:01:49,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 08a32e58e596f7f48d8374c3c2c4f4b3 2023-02-13 22:01:49,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 08a32e58e596f7f48d8374c3c2c4f4b3, disabling compactions & flushes 2023-02-13 22:01:49,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3. 2023-02-13 22:01:49,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3. 2023-02-13 22:01:49,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3. after waiting 0 ms 2023-02-13 22:01:49,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3. 2023-02-13 22:01:49,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:01:49,354 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3. 2023-02-13 22:01:49,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 08a32e58e596f7f48d8374c3c2c4f4b3: Waiting for close lock at 1676325709349Running coprocessor pre-close hooks at 1676325709349Disabling compacts and flushes for region at 1676325709349Disabling writes for close at 1676325709351 (+2 ms)Writing region close event to WAL at 1676325709352 (+1 ms)Running coprocessor post-close hooks at 1676325709352Closed at 1676325709354 (+2 ms) 2023-02-13 22:01:49,355 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:01:49,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing faa5a059f8b3120d55e00b3ba9ebb6b9, disabling compactions & flushes 2023-02-13 22:01:49,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:01:49,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:01:49,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. after waiting 0 ms 2023-02-13 22:01:49,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:01:49,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 08a32e58e596f7f48d8374c3c2c4f4b3 2023-02-13 22:01:49,360 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=08a32e58e596f7f48d8374c3c2c4f4b3, regionState=CLOSED 2023-02-13 22:01:49,361 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325709360"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676325709360"}]},"ts":"1676325709360"} 2023-02-13 22:01:49,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 22:01:49,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:01:49,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9. 2023-02-13 22:01:49,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for faa5a059f8b3120d55e00b3ba9ebb6b9: Waiting for close lock at 1676325709355Running coprocessor pre-close hooks at 1676325709355Disabling compacts and flushes for region at 1676325709355Disabling writes for close at 1676325709357 (+2 ms)Writing region close event to WAL at 1676325709358 (+1 ms)Running coprocessor post-close hooks at 1676325709364 (+6 ms)Closed at 1676325709365 (+1 ms) 2023-02-13 22:01:49,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-13 22:01:49,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 08a32e58e596f7f48d8374c3c2c4f4b3, server=jenkins-hbase20.apache.org,42047,1676325615467 in 166 msec 2023-02-13 22:01:49,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:01:49,367 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3, UNASSIGN in 174 msec 2023-02-13 22:01:49,368 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=faa5a059f8b3120d55e00b3ba9ebb6b9, regionState=CLOSED 2023-02-13 22:01:49,368 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676325709368"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325709368"}]},"ts":"1676325709368"} 2023-02-13 22:01:49,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-13 22:01:49,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure faa5a059f8b3120d55e00b3ba9ebb6b9, server=jenkins-hbase20.apache.org,37979,1676325615388 in 169 msec 2023-02-13 22:01:49,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-13 22:01:49,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=faa5a059f8b3120d55e00b3ba9ebb6b9, UNASSIGN in 179 msec 2023-02-13 22:01:49,373 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325709373"}]},"ts":"1676325709373"} 2023-02-13 22:01:49,373 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-13 22:01:49,374 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-13 22:01:49,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 191 msec 2023-02-13 22:01:49,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 22:01:49,492 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-13 22:01:49,492 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:49,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:49,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:49,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:49,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 22:01:49,499 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:01:49,501 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/recovered.edits] 2023-02-13 22:01:49,505 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/recovered.edits/7.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9/recovered.edits/7.seqid 2023-02-13 22:01:49,505 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/faa5a059f8b3120d55e00b3ba9ebb6b9 2023-02-13 22:01:49,505 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 22:01:49,507 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:49,509 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-13 22:01:49,511 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-13 22:01:49,512 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:49,512 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-13 22:01:49,512 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325709512"}]},"ts":"9223372036854775807"} 2023-02-13 22:01:49,512 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325709512"}]},"ts":"9223372036854775807"} 2023-02-13 22:01:49,514 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-13 22:01:49,514 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => faa5a059f8b3120d55e00b3ba9ebb6b9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676325644173.faa5a059f8b3120d55e00b3ba9ebb6b9.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 08a32e58e596f7f48d8374c3c2c4f4b3, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676325644173_0001.08a32e58e596f7f48d8374c3c2c4f4b3.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-13 22:01:49,514 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-13 22:01:49,515 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676325709514"}]},"ts":"9223372036854775807"} 2023-02-13 22:01:49,516 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-13 22:01:49,518 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:49,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 26 msec 2023-02-13 22:01:49,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 22:01:49,597 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-13 22:01:49,607 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=346 (was 411), OpenFileDescriptor=653 (was 852), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=108 (was 269), ProcessCount=162 (was 162), AvailableMemoryMB=7645 (was 7636) - AvailableMemoryMB LEAK? - 2023-02-13 22:01:49,619 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=347, OpenFileDescriptor=653, MaxFileDescriptor=60000, SystemLoadAverage=108, ProcessCount=162, AvailableMemoryMB=7645 2023-02-13 22:01:49,620 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:01:49,622 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:01:49,622 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-13 22:01:49,623 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:49,625 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:01:49,625 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:01:49,627 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:49,628 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703 empty. 2023-02-13 22:01:49,628 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:49,628 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 22:01:49,640 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-13 22:01:49,641 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2e173d609aef6dcecff6d15b746c1703, NAME => 'testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:01:49,647 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:01:49,648 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 2e173d609aef6dcecff6d15b746c1703, disabling compactions & flushes 2023-02-13 22:01:49,648 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:49,648 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:49,648 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. after waiting 0 ms 2023-02-13 22:01:49,648 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:49,648 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:49,648 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 2e173d609aef6dcecff6d15b746c1703: Waiting for close lock at 1676325709648Disabling compacts and flushes for region at 1676325709648Disabling writes for close at 1676325709648Writing region close event to WAL at 1676325709648Closed at 1676325709648 2023-02-13 22:01:49,650 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:01:49,650 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676325709650"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325709650"}]},"ts":"1676325709650"} 2023-02-13 22:01:49,652 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:01:49,652 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:01:49,653 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325709653"}]},"ts":"1676325709653"} 2023-02-13 22:01:49,654 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-13 22:01:49,655 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:01:49,656 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:01:49,656 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:01:49,656 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:01:49,657 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:01:49,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=2e173d609aef6dcecff6d15b746c1703, ASSIGN}] 2023-02-13 22:01:49,658 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=2e173d609aef6dcecff6d15b746c1703, ASSIGN 2023-02-13 22:01:49,659 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=2e173d609aef6dcecff6d15b746c1703, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:01:49,809 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:01:49,810 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=2e173d609aef6dcecff6d15b746c1703, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:49,811 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676325709810"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325709810"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325709810"}]},"ts":"1676325709810"} 2023-02-13 22:01:49,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 2e173d609aef6dcecff6d15b746c1703, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:01:49,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:49,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2e173d609aef6dcecff6d15b746c1703, NAME => 'testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:01:49,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:01:49,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:49,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:01:49,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:49,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:49,974 INFO [StoreOpener-2e173d609aef6dcecff6d15b746c1703-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:49,975 DEBUG [StoreOpener-2e173d609aef6dcecff6d15b746c1703-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1 2023-02-13 22:01:49,975 DEBUG [StoreOpener-2e173d609aef6dcecff6d15b746c1703-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1 2023-02-13 22:01:49,976 INFO [StoreOpener-2e173d609aef6dcecff6d15b746c1703-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2e173d609aef6dcecff6d15b746c1703 columnFamilyName cf1 2023-02-13 22:01:49,977 INFO [StoreOpener-2e173d609aef6dcecff6d15b746c1703-1] regionserver.HStore(310): Store=2e173d609aef6dcecff6d15b746c1703/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:49,977 INFO [StoreOpener-2e173d609aef6dcecff6d15b746c1703-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:49,979 DEBUG [StoreOpener-2e173d609aef6dcecff6d15b746c1703-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2 2023-02-13 22:01:49,979 DEBUG [StoreOpener-2e173d609aef6dcecff6d15b746c1703-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2 2023-02-13 22:01:49,980 INFO [StoreOpener-2e173d609aef6dcecff6d15b746c1703-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2e173d609aef6dcecff6d15b746c1703 columnFamilyName cf2 2023-02-13 22:01:49,981 INFO [StoreOpener-2e173d609aef6dcecff6d15b746c1703-1] regionserver.HStore(310): Store=2e173d609aef6dcecff6d15b746c1703/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:49,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:49,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:49,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:01:49,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:49,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:01:49,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 2e173d609aef6dcecff6d15b746c1703; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59963368, jitterRate=-0.10647618770599365}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:01:49,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 2e173d609aef6dcecff6d15b746c1703: Running coprocessor pre-open hook at 1676325709972Writing region info on filesystem at 1676325709972Initializing all the Stores at 1676325709973 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325709973Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325709973Cleaning up temporary data from old regions at 1676325709983 (+10 ms)Cleaning up detritus from prior splits at 1676325709984 (+1 ms)Running coprocessor post-open hooks at 1676325709987 (+3 ms)Region opened successfully at 1676325709987 2023-02-13 22:01:49,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703., pid=104, masterSystemTime=1676325709966 2023-02-13 22:01:49,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:49,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:49,989 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=2e173d609aef6dcecff6d15b746c1703, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:49,989 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676325709989"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325709989"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325709989"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325709989"}]},"ts":"1676325709989"} 2023-02-13 22:01:49,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-13 22:01:49,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 2e173d609aef6dcecff6d15b746c1703, server=jenkins-hbase20.apache.org,33193,1676325615429 in 177 msec 2023-02-13 22:01:49,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-13 22:01:49,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=2e173d609aef6dcecff6d15b746c1703, ASSIGN in 335 msec 2023-02-13 22:01:49,994 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:01:49,994 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325709994"}]},"ts":"1676325709994"} 2023-02-13 22:01:49,995 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-13 22:01:49,996 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:01:49,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 375 msec 2023-02-13 22:01:50,131 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:50,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 22:01:50,134 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 22:01:50,134 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,134 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 22:01:50,134 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 22:01:50,134 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,134 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,135 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,135 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,135 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,135 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,136 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,136 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,138 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,139 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,139 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,139 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:50,139 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:50,139 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:50,139 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. started... 2023-02-13 22:01:50,139 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e173d609aef6dcecff6d15b746c1703 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 22:01:50,140 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,140 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,140 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,140 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,140 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,140 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,147 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/a8507de2b7e64ae4b66158e2792aceb5 2023-02-13 22:01:50,158 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/69d94e5c555c41e692557971d76ace09 2023-02-13 22:01:50,162 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/a8507de2b7e64ae4b66158e2792aceb5 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/a8507de2b7e64ae4b66158e2792aceb5 2023-02-13 22:01:50,168 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/a8507de2b7e64ae4b66158e2792aceb5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:01:50,169 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/69d94e5c555c41e692557971d76ace09 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/69d94e5c555c41e692557971d76ace09 2023-02-13 22:01:50,174 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/69d94e5c555c41e692557971d76ace09, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:01:50,174 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 2e173d609aef6dcecff6d15b746c1703 in 35ms, sequenceid=6, compaction requested=false 2023-02-13 22:01:50,174 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-13 22:01:50,175 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e173d609aef6dcecff6d15b746c1703: 2023-02-13 22:01:50,175 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:50,175 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:50,175 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:50,175 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,175 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,175 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,176 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,176 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,176 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,176 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 22:01:50,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37eae592[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:50,178 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 22:01:50,178 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,179 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,179 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,179 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,179 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,179 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:50,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,179 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,179 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,179 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:50,179 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,179 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,179 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,179 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:50,179 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,179 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:50,180 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,180 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,180 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,180 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,180 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,180 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,180 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,180 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,180 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,180 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:01:50,183 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 22:01:50,183 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 22:01:50,183 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:50,183 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:50,183 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 22:01:50,183 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:50,184 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,184 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,184 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,184 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,184 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,184 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,184 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,185 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,185 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,186 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,188 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,188 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,188 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,195 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,195 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 22:01:50,196 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:50,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,195 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,196 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,196 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,196 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:50,196 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,196 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:50,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,196 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,197 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,197 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,197 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,197 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,297 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:50,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:50,322 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:50,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:50,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 22:01:50,324 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 22:01:50,325 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,325 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 22:01:50,325 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 22:01:50,325 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,325 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,328 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,328 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,328 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,328 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,328 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,328 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:50,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,330 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,330 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,330 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,330 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,330 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,330 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,331 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,331 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,331 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,331 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:50,331 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,332 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,332 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. started... 2023-02-13 22:01:50,332 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,332 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,332 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,332 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e173d609aef6dcecff6d15b746c1703 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:50,334 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,335 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,335 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,335 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,335 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,335 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,342 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/40989bc987ea47ae9ab253708f0fd487 2023-02-13 22:01:50,354 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/dbcc853265cf4046ac487ab777f23ff7 2023-02-13 22:01:50,357 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/40989bc987ea47ae9ab253708f0fd487 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/40989bc987ea47ae9ab253708f0fd487 2023-02-13 22:01:50,361 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/40989bc987ea47ae9ab253708f0fd487, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:01:50,362 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/dbcc853265cf4046ac487ab777f23ff7 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/dbcc853265cf4046ac487ab777f23ff7 2023-02-13 22:01:50,365 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/dbcc853265cf4046ac487ab777f23ff7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:01:50,366 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2e173d609aef6dcecff6d15b746c1703 in 34ms, sequenceid=19, compaction requested=false 2023-02-13 22:01:50,366 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e173d609aef6dcecff6d15b746c1703: 2023-02-13 22:01:50,366 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:50,366 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:50,367 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:50,367 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,367 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,367 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,368 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,368 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,368 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,368 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,369 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 22:01:50,369 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5334ed87[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:50,369 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 22:01:50,370 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,370 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,370 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,370 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,370 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,370 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,370 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:50,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,370 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:50,370 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,371 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,371 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,371 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:50,371 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:50,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,371 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,372 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,372 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,372 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,372 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,372 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,372 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,372 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:01:50,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 22:01:50,374 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 22:01:50,374 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:50,374 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:50,374 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 22:01:50,374 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:50,375 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,375 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,375 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,375 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,375 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,375 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,375 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,376 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,376 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,377 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,378 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,378 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,378 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,393 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,393 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,393 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,393 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,393 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,393 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 22:01:50,393 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:50,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,393 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,393 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,393 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,393 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,393 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:50,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,393 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,394 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:50,394 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,394 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,394 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,394 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,394 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,394 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,394 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,394 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,494 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:50,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:50,505 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:50,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:50,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 22:01:50,507 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 22:01:50,507 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,507 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 22:01:50,507 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 22:01:50,507 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,507 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,508 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,508 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,508 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,508 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,508 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,508 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,509 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,509 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:50,510 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:50,510 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. started... 2023-02-13 22:01:50,510 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e173d609aef6dcecff6d15b746c1703 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:50,510 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,511 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,511 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,511 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,511 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,511 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,518 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/680818ae73d54b799892751bc79ba425 2023-02-13 22:01:50,528 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/b85eb7186f474df689909ba764bd3f08 2023-02-13 22:01:50,532 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/680818ae73d54b799892751bc79ba425 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/680818ae73d54b799892751bc79ba425 2023-02-13 22:01:50,536 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/680818ae73d54b799892751bc79ba425, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:01:50,537 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/b85eb7186f474df689909ba764bd3f08 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/b85eb7186f474df689909ba764bd3f08 2023-02-13 22:01:50,539 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/b85eb7186f474df689909ba764bd3f08, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:01:50,540 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2e173d609aef6dcecff6d15b746c1703 in 30ms, sequenceid=32, compaction requested=true 2023-02-13 22:01:50,540 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e173d609aef6dcecff6d15b746c1703: 2023-02-13 22:01:50,540 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:50,540 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:50,540 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:50,540 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,541 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,541 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,542 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,542 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,542 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,542 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 22:01:50,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@edd0b30[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:50,543 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 22:01:50,543 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,544 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,544 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,544 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,544 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,544 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:50,544 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,544 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,544 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:50,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,544 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,544 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,544 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,544 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:50,544 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,544 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:50,545 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,545 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,545 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,545 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,545 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,545 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,545 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,545 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,545 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,545 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:01:50,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 22:01:50,547 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 22:01:50,547 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:50,547 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:50,547 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 22:01:50,547 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:50,548 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,548 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,548 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,548 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,548 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,548 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,549 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,549 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,549 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,549 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,551 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,551 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,551 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,554 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,554 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 22:01:50,554 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:50,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:50,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,554 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,654 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:50,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:50,666 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:50,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:50,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 22:01:50,667 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 22:01:50,667 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,667 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 22:01:50,667 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 22:01:50,668 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,668 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,669 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,669 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,669 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,669 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,669 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,669 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:50,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,670 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,670 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,670 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,670 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,670 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,670 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,670 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,671 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,670 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,671 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,671 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,671 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,671 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,671 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,671 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,671 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:50,671 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:50,671 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:50,671 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. started... 2023-02-13 22:01:50,671 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e173d609aef6dcecff6d15b746c1703 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:50,671 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,672 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,672 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,672 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,672 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,672 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,678 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/1bfb091f09804cae9c0bef5c3c6179c3 2023-02-13 22:01:50,688 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/fd54c1b211734e7da6a598b14dbecf5d 2023-02-13 22:01:50,691 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/1bfb091f09804cae9c0bef5c3c6179c3 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/1bfb091f09804cae9c0bef5c3c6179c3 2023-02-13 22:01:50,694 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/1bfb091f09804cae9c0bef5c3c6179c3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:01:50,695 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/fd54c1b211734e7da6a598b14dbecf5d as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/fd54c1b211734e7da6a598b14dbecf5d 2023-02-13 22:01:50,698 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/fd54c1b211734e7da6a598b14dbecf5d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:01:50,699 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2e173d609aef6dcecff6d15b746c1703 in 28ms, sequenceid=45, compaction requested=true 2023-02-13 22:01:50,699 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e173d609aef6dcecff6d15b746c1703: 2023-02-13 22:01:50,699 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:50,699 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:50,699 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:50,699 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,699 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,699 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,700 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,700 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,700 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,701 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 22:01:50,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75648cd2[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:50,702 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 22:01:50,702 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,703 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,703 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,703 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,703 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,703 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,703 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,703 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,703 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:50,703 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,703 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:50,703 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,704 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,704 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,704 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:50,704 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,704 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,704 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,704 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,704 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,704 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,705 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,705 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,705 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,705 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:01:50,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 22:01:50,708 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 22:01:50,708 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:50,708 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:50,708 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 22:01:50,708 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:50,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,709 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,710 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,710 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,712 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,712 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,712 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,714 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 22:01:50,714 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:50,714 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:50,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,715 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:50,714 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,715 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,715 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,715 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,715 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,715 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,715 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,815 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:50,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:50,830 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:50,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:50,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 22:01:50,832 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 22:01:50,832 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,832 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 22:01:50,833 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 22:01:50,833 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,833 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,834 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,834 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,834 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,834 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,835 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,835 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,835 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:50,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,835 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,835 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,835 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,836 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:50,836 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,836 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,836 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,836 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,836 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:50,836 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,836 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,836 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,836 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:50,836 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,837 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:50,837 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,837 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,837 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:50,837 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,837 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:50,837 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:50,837 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:50,837 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:50,837 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:50,837 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. started... 2023-02-13 22:01:50,837 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,837 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e173d609aef6dcecff6d15b746c1703 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:50,838 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,838 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,838 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,838 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,838 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,848 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/0a8104d35121436d8704dac6fe72de63 2023-02-13 22:01:50,862 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/4d3afe62a9274841ab3a53549687770a 2023-02-13 22:01:50,866 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/0a8104d35121436d8704dac6fe72de63 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/0a8104d35121436d8704dac6fe72de63 2023-02-13 22:01:50,870 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/0a8104d35121436d8704dac6fe72de63, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:01:50,871 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/4d3afe62a9274841ab3a53549687770a as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/4d3afe62a9274841ab3a53549687770a 2023-02-13 22:01:50,876 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/4d3afe62a9274841ab3a53549687770a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:01:50,877 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2e173d609aef6dcecff6d15b746c1703 in 40ms, sequenceid=58, compaction requested=true 2023-02-13 22:01:50,877 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e173d609aef6dcecff6d15b746c1703: 2023-02-13 22:01:50,877 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:50,877 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:50,877 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:50,877 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,877 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:50,877 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:50,879 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,879 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,880 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,880 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 22:01:50,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15423199[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:50,882 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 22:01:50,882 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,883 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,883 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,883 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,884 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,884 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,884 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,884 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:50,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,884 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,884 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:50,884 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,884 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:50,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,884 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:50,884 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:50,884 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:50,885 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,885 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,885 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,885 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,885 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,885 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,885 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,885 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:50,886 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,886 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:50,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:01:50,890 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 22:01:50,890 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 22:01:50,890 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:50,890 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:50,890 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 22:01:50,890 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:50,895 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,895 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,895 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:50,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:50,896 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,895 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,895 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,896 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:50,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,896 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,897 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:50,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:50,898 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:50,899 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:50,900 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,900 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,902 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,903 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:50,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,903 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 22:01:50,903 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:50,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:50,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:50,903 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:50,904 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:50,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:50,904 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:50,904 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:50,904 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:50,904 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:50,904 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,004 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:51,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:51,026 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:51,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:51,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 22:01:51,028 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 22:01:51,028 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:51,028 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 22:01:51,028 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 22:01:51,029 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,029 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:51,030 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,030 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:51,030 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:51,030 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:51,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:51,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:51,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:51,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:51,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:51,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:51,030 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,030 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:51,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:51,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:51,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:51,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:51,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:51,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:51,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:51,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:51,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:51,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:51,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:51,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:51,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:51,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:51,032 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:51,032 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:51,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:51,032 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:51,032 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:51,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:51,033 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:51,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 22:01:51,033 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. started... 2023-02-13 22:01:51,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:51,033 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e173d609aef6dcecff6d15b746c1703 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:51,033 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:51,034 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,034 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,034 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,034 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:51,035 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,035 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:51,045 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/6e3a403048994284845dcae6397fb2e6 2023-02-13 22:01:51,058 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/d97db798aadf47d1ad991b92c84ce4d7 2023-02-13 22:01:51,061 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/6e3a403048994284845dcae6397fb2e6 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/6e3a403048994284845dcae6397fb2e6 2023-02-13 22:01:51,064 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/6e3a403048994284845dcae6397fb2e6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:01:51,065 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/d97db798aadf47d1ad991b92c84ce4d7 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/d97db798aadf47d1ad991b92c84ce4d7 2023-02-13 22:01:51,068 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/d97db798aadf47d1ad991b92c84ce4d7, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:01:51,069 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2e173d609aef6dcecff6d15b746c1703 in 36ms, sequenceid=71, compaction requested=true 2023-02-13 22:01:51,069 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e173d609aef6dcecff6d15b746c1703: 2023-02-13 22:01:51,069 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:51,069 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:51,069 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:51,069 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,070 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 22:01:51,070 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 22:01:51,071 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,071 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:51,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:51,071 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,071 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:51,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:51,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:51,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:51,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:51,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:51,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:51,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 22:01:51,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61eeae63[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:51,073 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 22:01:51,073 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,074 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,074 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,074 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,074 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:51,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,074 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:51,074 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,074 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:51,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,074 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:51,074 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:51,075 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 22:01:51,075 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:51,075 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:51,075 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 22:01:51,075 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:51,075 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:51,076 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:51,076 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:51,076 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:51,076 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 22:01:51,076 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:51,076 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,076 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:51,076 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:51,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:51,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:51,076 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 22:01:51,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:51,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:51,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:51,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:51,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:51,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:01:51,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 22:01:51,080 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 22:01:51,080 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:51,080 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:51,081 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 22:01:51,081 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:51,084 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,084 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,084 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,084 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,084 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:51,084 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:51,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,084 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:51,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:51,085 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:51,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:51,086 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:51,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:51,086 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:51,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:51,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:51,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:51,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,088 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:51,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:51,089 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:51,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 22:01:51,089 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:51,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:51,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,095 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,095 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:51,095 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:51,095 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:51,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:51,095 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:51,095 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:51,095 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:51,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:51,095 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 22:01:51,096 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:51,095 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:51,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:51,096 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:51,096 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:51,095 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:51,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:51,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:51,096 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:51,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:51,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 22:01:51,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:51,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:51,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:51,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 22:01:51,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 22:01:51,197 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:51,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:51,215 INFO [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,216 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/40989bc987ea47ae9ab253708f0fd487 because midkey is the same as first or last row 2023-02-13 22:01:51,217 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/dbcc853265cf4046ac487ab777f23ff7 because midkey is the same as first or last row 2023-02-13 22:01:51,217 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.HRegion(2744): Flushing 2e173d609aef6dcecff6d15b746c1703 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 22:01:51,229 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/0071d4fc23de46299190864a208e60da 2023-02-13 22:01:51,244 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/93e11f81882547beaf7b12ea67140a34 2023-02-13 22:01:51,251 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf1/0071d4fc23de46299190864a208e60da as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/0071d4fc23de46299190864a208e60da 2023-02-13 22:01:51,258 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/0071d4fc23de46299190864a208e60da, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:01:51,259 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/.tmp/cf2/93e11f81882547beaf7b12ea67140a34 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/93e11f81882547beaf7b12ea67140a34 2023-02-13 22:01:51,264 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/93e11f81882547beaf7b12ea67140a34, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:01:51,265 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2e173d609aef6dcecff6d15b746c1703 in 48ms, sequenceid=82, compaction requested=true 2023-02-13 22:01:51,266 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.HRegion(2445): Flush status journal for 2e173d609aef6dcecff6d15b746c1703: 2023-02-13 22:01:51,266 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/40989bc987ea47ae9ab253708f0fd487 because midkey is the same as first or last row 2023-02-13 22:01:51,266 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33193] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/dbcc853265cf4046ac487ab777f23ff7 because midkey is the same as first or last row 2023-02-13 22:01:51,266 DEBUG [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,266 DEBUG [Listener at localhost.localdomain/42295] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-13 22:01:51,267 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:01:51,269 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-13 22:01:51,269 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testInvalidSplitKey 2023-02-13 22:01:51,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:51,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 22:01:51,273 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325711273"}]},"ts":"1676325711273"} 2023-02-13 22:01:51,275 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-13 22:01:51,276 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-13 22:01:51,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=2e173d609aef6dcecff6d15b746c1703, UNASSIGN}] 2023-02-13 22:01:51,278 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=2e173d609aef6dcecff6d15b746c1703, UNASSIGN 2023-02-13 22:01:51,278 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=2e173d609aef6dcecff6d15b746c1703, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:51,279 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676325711278"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325711278"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325711278"}]},"ts":"1676325711278"} 2023-02-13 22:01:51,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 2e173d609aef6dcecff6d15b746c1703, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:01:51,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 22:01:51,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:51,434 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 2e173d609aef6dcecff6d15b746c1703, disabling compactions & flushes 2023-02-13 22:01:51,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:51,434 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:51,434 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. after waiting 0 ms 2023-02-13 22:01:51,434 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:51,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 22:01:51,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:01:51,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703. 2023-02-13 22:01:51,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 2e173d609aef6dcecff6d15b746c1703: Waiting for close lock at 1676325711434Running coprocessor pre-close hooks at 1676325711434Disabling compacts and flushes for region at 1676325711434Disabling writes for close at 1676325711434Writing region close event to WAL at 1676325711437 (+3 ms)Running coprocessor post-close hooks at 1676325711445 (+8 ms)Closed at 1676325711445 2023-02-13 22:01:51,447 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:51,447 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=2e173d609aef6dcecff6d15b746c1703, regionState=CLOSED 2023-02-13 22:01:51,448 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676325711447"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325711447"}]},"ts":"1676325711447"} 2023-02-13 22:01:51,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-13 22:01:51,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 2e173d609aef6dcecff6d15b746c1703, server=jenkins-hbase20.apache.org,33193,1676325615429 in 169 msec 2023-02-13 22:01:51,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-13 22:01:51,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=2e173d609aef6dcecff6d15b746c1703, UNASSIGN in 174 msec 2023-02-13 22:01:51,454 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325711454"}]},"ts":"1676325711454"} 2023-02-13 22:01:51,455 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-13 22:01:51,456 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-13 22:01:51,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 188 msec 2023-02-13 22:01:51,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 22:01:51,579 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-13 22:01:51,581 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testInvalidSplitKey 2023-02-13 22:01:51,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:51,585 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:51,586 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:51,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 22:01:51,592 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:51,596 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/recovered.edits] 2023-02-13 22:01:51,600 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/0071d4fc23de46299190864a208e60da to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/0071d4fc23de46299190864a208e60da 2023-02-13 22:01:51,601 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/0a8104d35121436d8704dac6fe72de63 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/0a8104d35121436d8704dac6fe72de63 2023-02-13 22:01:51,602 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/1bfb091f09804cae9c0bef5c3c6179c3 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/1bfb091f09804cae9c0bef5c3c6179c3 2023-02-13 22:01:51,604 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/40989bc987ea47ae9ab253708f0fd487 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/40989bc987ea47ae9ab253708f0fd487 2023-02-13 22:01:51,605 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/680818ae73d54b799892751bc79ba425 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/680818ae73d54b799892751bc79ba425 2023-02-13 22:01:51,606 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/6e3a403048994284845dcae6397fb2e6 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/6e3a403048994284845dcae6397fb2e6 2023-02-13 22:01:51,607 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/a8507de2b7e64ae4b66158e2792aceb5 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf1/a8507de2b7e64ae4b66158e2792aceb5 2023-02-13 22:01:51,610 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/4d3afe62a9274841ab3a53549687770a to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/4d3afe62a9274841ab3a53549687770a 2023-02-13 22:01:51,611 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/69d94e5c555c41e692557971d76ace09 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/69d94e5c555c41e692557971d76ace09 2023-02-13 22:01:51,613 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/93e11f81882547beaf7b12ea67140a34 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/93e11f81882547beaf7b12ea67140a34 2023-02-13 22:01:51,614 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/b85eb7186f474df689909ba764bd3f08 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/b85eb7186f474df689909ba764bd3f08 2023-02-13 22:01:51,615 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/d97db798aadf47d1ad991b92c84ce4d7 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/d97db798aadf47d1ad991b92c84ce4d7 2023-02-13 22:01:51,616 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/dbcc853265cf4046ac487ab777f23ff7 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/dbcc853265cf4046ac487ab777f23ff7 2023-02-13 22:01:51,617 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/fd54c1b211734e7da6a598b14dbecf5d to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/cf2/fd54c1b211734e7da6a598b14dbecf5d 2023-02-13 22:01:51,619 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/recovered.edits/85.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703/recovered.edits/85.seqid 2023-02-13 22:01:51,620 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testInvalidSplitKey/2e173d609aef6dcecff6d15b746c1703 2023-02-13 22:01:51,620 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 22:01:51,621 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:51,622 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-13 22:01:51,626 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-13 22:01:51,627 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:51,627 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-13 22:01:51,627 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325711627"}]},"ts":"9223372036854775807"} 2023-02-13 22:01:51,628 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 22:01:51,628 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2e173d609aef6dcecff6d15b746c1703, NAME => 'testInvalidSplitKey,,1676325709622.2e173d609aef6dcecff6d15b746c1703.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 22:01:51,628 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-13 22:01:51,628 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676325711628"}]},"ts":"9223372036854775807"} 2023-02-13 22:01:51,630 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-13 22:01:51,631 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:51,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 50 msec 2023-02-13 22:01:51,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 22:01:51,689 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-13 22:01:51,706 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=368 (was 347) - Thread LEAK? -, OpenFileDescriptor=740 (was 653) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=115 (was 108) - SystemLoadAverage LEAK? -, ProcessCount=162 (was 162), AvailableMemoryMB=7643 (was 7645) 2023-02-13 22:01:51,723 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=368, OpenFileDescriptor=740, MaxFileDescriptor=60000, SystemLoadAverage=115, ProcessCount=162, AvailableMemoryMB=7642 2023-02-13 22:01:51,724 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:01:51,725 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:01:51,726 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-13 22:01:51,727 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:51,728 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:01:51,728 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:01:51,729 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:51,730 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369 empty. 2023-02-13 22:01:51,730 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:51,730 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 22:01:51,738 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 22:01:51,739 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => fe45e04d204e94f79d1eca4598523369, NAME => 'testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:01:51,748 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:01:51,748 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing fe45e04d204e94f79d1eca4598523369, disabling compactions & flushes 2023-02-13 22:01:51,749 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:51,749 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:51,749 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. after waiting 0 ms 2023-02-13 22:01:51,749 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:51,749 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:51,749 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for fe45e04d204e94f79d1eca4598523369: Waiting for close lock at 1676325711748Disabling compacts and flushes for region at 1676325711748Disabling writes for close at 1676325711749 (+1 ms)Writing region close event to WAL at 1676325711749Closed at 1676325711749 2023-02-13 22:01:51,752 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:01:51,753 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325711753"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325711753"}]},"ts":"1676325711753"} 2023-02-13 22:01:51,754 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:01:51,755 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:01:51,755 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325711755"}]},"ts":"1676325711755"} 2023-02-13 22:01:51,757 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 22:01:51,759 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:01:51,760 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:01:51,760 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:01:51,760 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:01:51,760 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:01:51,761 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, ASSIGN}] 2023-02-13 22:01:51,762 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, ASSIGN 2023-02-13 22:01:51,763 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,42047,1676325615467; forceNewPlan=false, retain=false 2023-02-13 22:01:51,913 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:01:51,915 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=fe45e04d204e94f79d1eca4598523369, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:51,916 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325711915"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325711915"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325711915"}]},"ts":"1676325711915"} 2023-02-13 22:01:51,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:01:52,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fe45e04d204e94f79d1eca4598523369, NAME => 'testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:01:52,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:01:52,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:52,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:01:52,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:52,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:52,083 INFO [StoreOpener-fe45e04d204e94f79d1eca4598523369-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:52,087 DEBUG [StoreOpener-fe45e04d204e94f79d1eca4598523369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1 2023-02-13 22:01:52,087 DEBUG [StoreOpener-fe45e04d204e94f79d1eca4598523369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1 2023-02-13 22:01:52,088 INFO [StoreOpener-fe45e04d204e94f79d1eca4598523369-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fe45e04d204e94f79d1eca4598523369 columnFamilyName cf1 2023-02-13 22:01:52,089 INFO [StoreOpener-fe45e04d204e94f79d1eca4598523369-1] regionserver.HStore(310): Store=fe45e04d204e94f79d1eca4598523369/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:52,089 INFO [StoreOpener-fe45e04d204e94f79d1eca4598523369-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:52,091 DEBUG [StoreOpener-fe45e04d204e94f79d1eca4598523369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2 2023-02-13 22:01:52,091 DEBUG [StoreOpener-fe45e04d204e94f79d1eca4598523369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2 2023-02-13 22:01:52,092 INFO [StoreOpener-fe45e04d204e94f79d1eca4598523369-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fe45e04d204e94f79d1eca4598523369 columnFamilyName cf2 2023-02-13 22:01:52,092 INFO [StoreOpener-fe45e04d204e94f79d1eca4598523369-1] regionserver.HStore(310): Store=fe45e04d204e94f79d1eca4598523369/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:52,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:52,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:52,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:01:52,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:52,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:01:52,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened fe45e04d204e94f79d1eca4598523369; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60626529, jitterRate=-0.09659431874752045}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:01:52,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for fe45e04d204e94f79d1eca4598523369: Running coprocessor pre-open hook at 1676325712081Writing region info on filesystem at 1676325712081Initializing all the Stores at 1676325712083 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325712083Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325712083Cleaning up temporary data from old regions at 1676325712095 (+12 ms)Cleaning up detritus from prior splits at 1676325712095Running coprocessor post-open hooks at 1676325712101 (+6 ms)Region opened successfully at 1676325712101 2023-02-13 22:01:52,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369., pid=111, masterSystemTime=1676325712072 2023-02-13 22:01:52,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,104 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=fe45e04d204e94f79d1eca4598523369, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,104 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325712104"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325712104"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325712104"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325712104"}]},"ts":"1676325712104"} 2023-02-13 22:01:52,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-13 22:01:52,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 in 186 msec 2023-02-13 22:01:52,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-13 22:01:52,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, ASSIGN in 347 msec 2023-02-13 22:01:52,110 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:01:52,111 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325712110"}]},"ts":"1676325712110"} 2023-02-13 22:01:52,112 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 22:01:52,113 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:01:52,115 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 388 msec 2023-02-13 22:01:52,181 ERROR [BP-109365581-148.251.75.209-1676325611364 heartbeating to localhost.localdomain/127.0.0.1:37463] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-109365581-148.251.75.209-1676325611364 (Datanode Uuid 3f4de7bc-de25-42a6-88c4-90335eaeb4a2) service to localhost.localdomain/127.0.0.1:37463 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 22:01:52,242 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:52,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 22:01:52,244 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:52,244 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,244 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 22:01:52,244 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:01:52,244 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,244 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,245 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,245 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,245 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,245 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,246 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,246 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:52,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:52,247 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:52,247 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,247 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. started... 2023-02-13 22:01:52,248 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe45e04d204e94f79d1eca4598523369 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 22:01:52,248 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,249 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,249 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,249 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,249 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,249 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,256 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/efa49219a1404a4f8f21adbde804fa14 2023-02-13 22:01:52,266 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/d22214c623124e9faa5e17aa1f6709a7 2023-02-13 22:01:52,271 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/efa49219a1404a4f8f21adbde804fa14 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14 2023-02-13 22:01:52,275 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:01:52,276 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/d22214c623124e9faa5e17aa1f6709a7 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7 2023-02-13 22:01:52,280 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:01:52,281 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for fe45e04d204e94f79d1eca4598523369 in 33ms, sequenceid=6, compaction requested=false 2023-02-13 22:01:52,281 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:52,281 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe45e04d204e94f79d1eca4598523369: 2023-02-13 22:01:52,282 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,282 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:52,282 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:52,282 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,282 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,282 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,283 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,283 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,284 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,284 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 22:01:52,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70cade5[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:52,286 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:01:52,286 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,287 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,287 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,287 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:52,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,287 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,287 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,288 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,289 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,289 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,289 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,289 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,289 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,289 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:01:52,292 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:01:52,292 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 22:01:52,292 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:52,292 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:52,292 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,292 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:52,293 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,293 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,293 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,293 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,293 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,293 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,293 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,293 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,294 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,294 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,295 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,295 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,296 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,297 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,297 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,297 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,297 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,297 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,297 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,297 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:52,297 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:52,297 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,297 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,297 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,298 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:52,297 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,298 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:52,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,298 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,398 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:52,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:52,420 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:52,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:52,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 22:01:52,422 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:52,422 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,422 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 22:01:52,422 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:01:52,423 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,423 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,424 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,424 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,424 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,424 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,425 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,425 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:52,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:52,427 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:52,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,427 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:52,427 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. started... 2023-02-13 22:01:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,428 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,428 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,428 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe45e04d204e94f79d1eca4598523369 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:52,429 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,430 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,430 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,430 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,431 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,431 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,438 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/88c5fddce6074a2787a34282e42fe7f1 2023-02-13 22:01:52,452 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/21da36534a1140dea813aacdae9c8520 2023-02-13 22:01:52,457 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/88c5fddce6074a2787a34282e42fe7f1 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1 2023-02-13 22:01:52,460 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:01:52,461 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/21da36534a1140dea813aacdae9c8520 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520 2023-02-13 22:01:52,464 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:01:52,464 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fe45e04d204e94f79d1eca4598523369 in 36ms, sequenceid=19, compaction requested=false 2023-02-13 22:01:52,465 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe45e04d204e94f79d1eca4598523369: 2023-02-13 22:01:52,465 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,465 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:52,465 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:52,465 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,465 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,465 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,468 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,468 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,468 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,468 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 22:01:52,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@565fb3ca[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:52,470 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:01:52,470 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,470 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,470 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,470 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,471 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,471 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,471 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,471 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:52,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,471 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:52,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,471 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,471 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,471 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,471 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,471 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:52,471 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:52,472 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,472 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,472 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,472 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,472 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,472 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,472 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,473 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,473 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,473 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:01:52,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:01:52,476 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 22:01:52,477 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:52,477 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:52,477 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,477 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:52,495 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,495 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,495 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,495 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,495 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,496 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,496 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,497 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,498 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,501 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,501 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,502 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,506 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,506 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,506 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,506 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,506 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,506 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,506 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:52,507 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:52,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,506 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,506 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,506 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,507 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,607 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:52,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:52,629 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:52,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:52,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 22:01:52,631 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:52,631 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,631 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 22:01:52,631 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:01:52,632 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,632 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,633 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,633 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,633 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,634 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:52,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,635 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:52,635 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,635 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:52,635 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,635 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:52,636 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:52,636 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,636 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. started... 2023-02-13 22:01:52,636 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe45e04d204e94f79d1eca4598523369 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:52,637 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,637 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,637 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,637 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,638 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,638 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,646 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/5d52eeef35a94f32bb7b454c4e087701 2023-02-13 22:01:52,662 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/e67902f7e0fb432c8377a88854bde947 2023-02-13 22:01:52,668 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/5d52eeef35a94f32bb7b454c4e087701 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701 2023-02-13 22:01:52,673 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:01:52,674 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/e67902f7e0fb432c8377a88854bde947 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947 2023-02-13 22:01:52,679 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:01:52,680 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fe45e04d204e94f79d1eca4598523369 in 44ms, sequenceid=32, compaction requested=true 2023-02-13 22:01:52,680 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe45e04d204e94f79d1eca4598523369: 2023-02-13 22:01:52,681 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,681 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:52,681 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:52,681 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,681 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,681 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,682 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,682 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,683 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,683 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 22:01:52,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52158e61[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:52,686 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:01:52,686 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,687 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,687 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,687 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,687 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,687 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,687 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:52,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,687 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,688 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:52,687 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,688 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,688 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:52,688 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,688 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,688 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:52,689 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,689 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,689 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:01:52,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:01:52,695 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 22:01:52,695 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:52,695 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:52,695 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,695 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:52,701 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,701 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,702 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,701 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,701 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,701 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,702 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,702 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,702 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,703 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,705 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,705 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,705 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,708 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,708 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:52,708 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:52,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,708 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution because node does not exist (not an error) 2023-02-13 22:01:52,708 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,709 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:52,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,709 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:52,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,709 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,809 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:52,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:52,824 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:52,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:52,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 22:01:52,826 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:52,826 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,826 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 22:01:52,826 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:01:52,827 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,827 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,828 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,828 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,828 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,828 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,829 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,829 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:52,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:52,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,831 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,830 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,831 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:52,831 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:52,831 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,831 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:52,831 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,831 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,831 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:52,831 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,831 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,831 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,831 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:52,832 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:52,832 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. started... 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,832 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe45e04d204e94f79d1eca4598523369 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,832 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,837 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,837 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,837 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,837 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,837 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,837 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,842 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/80bb277e72704b8c99cdf87e3fd889e0 2023-02-13 22:01:52,857 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/10a3bfd393ee45b7b9a65d3bb7040b30 2023-02-13 22:01:52,860 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/80bb277e72704b8c99cdf87e3fd889e0 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0 2023-02-13 22:01:52,863 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:01:52,864 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/10a3bfd393ee45b7b9a65d3bb7040b30 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30 2023-02-13 22:01:52,867 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:01:52,867 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fe45e04d204e94f79d1eca4598523369 in 35ms, sequenceid=45, compaction requested=true 2023-02-13 22:01:52,867 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe45e04d204e94f79d1eca4598523369: 2023-02-13 22:01:52,868 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:52,868 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:52,868 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:52,868 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,868 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:52,868 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:52,869 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,869 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,869 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,869 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:52,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 22:01:52,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@469cbff6[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:52,871 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:01:52,871 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,872 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,872 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,872 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,872 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,872 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:52,872 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,873 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,873 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:52,873 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,873 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,873 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:52,873 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:52,873 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:52,873 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:52,873 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,874 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,874 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,874 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,874 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,874 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,874 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,874 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:52,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,874 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,874 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:52,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:01:52,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:01:52,877 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 22:01:52,877 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:52,877 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:52,877 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 22:01:52,878 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:52,879 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,879 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:52,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:52,879 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:52,879 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,880 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:52,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,881 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,882 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:52,882 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:52,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,887 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,887 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,887 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,887 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,887 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:52,887 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,887 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:52,887 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:52,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,887 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,887 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,887 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:52,887 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,887 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:52,888 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,888 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:52,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,888 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:52,888 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:52,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:52,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:52,888 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,888 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:52,888 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:52,889 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,889 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:52,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:52,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:52,988 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:52,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:53,012 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:53,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:53,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 22:01:53,014 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:53,014 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:53,014 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 22:01:53,014 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:01:53,015 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,015 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,016 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,016 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,016 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,016 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,016 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,016 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:53,017 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,017 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:53,018 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:53,018 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:53,018 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:53,018 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:53,018 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:53,018 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:53,018 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:53,018 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:53,019 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. started... 2023-02-13 22:01:53,019 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe45e04d204e94f79d1eca4598523369 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,021 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,021 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:53,021 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,021 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:53,029 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/6884cbf58a2446e9a50a05fb5832f51d 2023-02-13 22:01:53,044 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/27319b9865844539846840838d43a0f9 2023-02-13 22:01:53,048 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/6884cbf58a2446e9a50a05fb5832f51d as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d 2023-02-13 22:01:53,054 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:01:53,055 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/27319b9865844539846840838d43a0f9 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9 2023-02-13 22:01:53,060 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:01:53,061 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fe45e04d204e94f79d1eca4598523369 in 42ms, sequenceid=58, compaction requested=true 2023-02-13 22:01:53,061 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe45e04d204e94f79d1eca4598523369: 2023-02-13 22:01:53,061 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:53,061 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:53,061 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:53,061 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,061 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:53,061 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:53,062 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:53,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:53,063 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:53,063 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,063 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:53,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:53,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:53,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 22:01:53,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@249d4145[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:53,065 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:01:53,066 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,066 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,066 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,066 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,067 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,067 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:53,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,067 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:53,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,067 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:53,067 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:53,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,067 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:53,067 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:53,067 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:53,067 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:53,067 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:53,067 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:53,068 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:53,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:53,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:53,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:53,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:53,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:53,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:01:53,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:01:53,071 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 22:01:53,071 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:53,071 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:53,071 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 22:01:53,071 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:53,072 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,072 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,072 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,072 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,072 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:53,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:53,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,072 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,072 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,072 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:53,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,073 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,073 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:53,075 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,075 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,075 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,093 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:53,093 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:53,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:53,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,093 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:53,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,094 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,094 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:53,094 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,094 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,094 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,194 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:53,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:53,214 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:53,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:53,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 22:01:53,216 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:53,216 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:53,216 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 22:01:53,216 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 22:01:53,216 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,216 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,217 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,217 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,217 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,217 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,218 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,218 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:53,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,218 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,218 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,218 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 22:01:53,219 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:53,219 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:53,219 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:53,219 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 22:01:53,219 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:53,219 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:53,219 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:53,219 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 22:01:53,219 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:53,219 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:53,219 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:53,220 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:53,219 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:53,220 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:53,220 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 22:01:53,220 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:53,220 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:53,220 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:53,220 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:53,220 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:53,220 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. started... 2023-02-13 22:01:53,220 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe45e04d204e94f79d1eca4598523369 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:53,221 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,221 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,221 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,221 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:53,221 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,221 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:53,230 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/b8905f0ed6fa408bb58db5cfc3672b9c 2023-02-13 22:01:53,244 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/240f909ea4d54bda9771214216b0c1bc 2023-02-13 22:01:53,252 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/b8905f0ed6fa408bb58db5cfc3672b9c as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c 2023-02-13 22:01:53,258 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:01:53,260 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/240f909ea4d54bda9771214216b0c1bc as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc 2023-02-13 22:01:53,266 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:01:53,267 INFO [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fe45e04d204e94f79d1eca4598523369 in 47ms, sequenceid=71, compaction requested=true 2023-02-13 22:01:53,267 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe45e04d204e94f79d1eca4598523369: 2023-02-13 22:01:53,267 DEBUG [rs(jenkins-hbase20.apache.org,42047,1676325615467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:53,268 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:53,268 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:53,268 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,268 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 22:01:53,268 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 22:01:53,269 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,269 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:53,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:53,270 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,270 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:53,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:53,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:53,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:53,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 22:01:53,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ff777ff[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:53,274 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 22:01:53,274 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,279 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,279 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,279 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,279 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:53,279 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:53,279 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,279 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:53,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,279 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:53,279 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:53,280 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:53,280 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:53,280 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 22:01:53,280 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 22:01:53,280 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:53,281 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,281 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:53,281 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:53,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:53,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:53,281 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:53,282 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:53,281 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:53,282 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:53,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:53,282 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 22:01:53,283 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:53,283 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 22:01:53,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:53,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:53,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,42047,1676325615467': 2023-02-13 22:01:53,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 22:01:53,288 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 22:01:53,289 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:53,289 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:53,289 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 22:01:53,289 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:53,296 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,296 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,296 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,296 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,296 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:53,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,296 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,297 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,297 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:53,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:53,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:53,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,300 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,301 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:53,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 22:01:53,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,303 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,304 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,304 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,307 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 22:01:53,307 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:53,308 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:53,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:53,307 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:53,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:53,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,307 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,308 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:53,308 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:53,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,308 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,309 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,309 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:53,309 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:53,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:53,309 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 22:01:53,408 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:53,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:53,675 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 22:01:53,677 DEBUG [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:53,679 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,682 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,682 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,682 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,684 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:53,684 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:53,685 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:53,930 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,930 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:53,930 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:53,931 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:53,931 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-13 22:01:53,932 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:53,932 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:53,932 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:53,932 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:53,934 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:01:53,939 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 22:01:53,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:53,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:53,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:53,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:53,939 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:53,940 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:53,940 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:53,940 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:53,940 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:53,940 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:53,940 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:53,940 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:53,941 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:53,941 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:53,941 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:53,941 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:53,941 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:53,941 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:53,941 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:53,941 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:53,942 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:53,942 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:53,942 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:53,942 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:53,942 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:53,942 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:53,942 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:53,943 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:53,943 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:53,943 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:53,943 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:53,943 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:53,943 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:53,943 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:53,944 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,944 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:53,944 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,944 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,944 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,944 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 22:01:53,944 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:53,945 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:53,945 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325743945 2023-02-13 22:01:53,946 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,946 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:53,949 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:53,949 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:53,949 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:53,949 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325773949 2023-02-13 22:01:53,949 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325833949 2023-02-13 22:01:53,950 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:01:53,950 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:53,952 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:53,952 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:53,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:53,952 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:54,200 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,200 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:54,200 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:54,201 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:54,201 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:01:54,201 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:54,202 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:54,202 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:54,202 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:54,204 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:01:54,208 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 22:01:54,209 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:54,209 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:54,210 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:54,210 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:54,211 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:54,211 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:54,211 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:54,211 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:54,211 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:54,211 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:54,211 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:54,211 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:54,212 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:54,212 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:54,212 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:54,212 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:54,212 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:54,212 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:54,212 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:54,212 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:54,213 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:54,213 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:54,213 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:54,213 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:54,213 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:54,213 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,213 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,213 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:54,214 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,214 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,214 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:54,214 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:54,214 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:54,214 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:54,214 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,214 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:54,214 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,214 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,215 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,215 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:01:54,215 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:54,216 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:54,216 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325744216 2023-02-13 22:01:54,218 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,218 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:54,220 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:54,220 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:54,220 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:54,221 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325774221 2023-02-13 22:01:54,221 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325834221 2023-02-13 22:01:54,221 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:01:54,221 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:54,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN}] 2023-02-13 22:01:54,223 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:54,223 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:54,223 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:54,471 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,471 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:54,471 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:54,471 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:54,471 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:01:54,471 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:54,471 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:54,471 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:54,471 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:54,472 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:01:54,473 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 22:01:54,473 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:54,474 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:54,474 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:54,474 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:54,474 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:54,474 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:54,474 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:54,474 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:54,475 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:54,475 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:54,475 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:54,475 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:54,475 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:54,476 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:54,476 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:54,476 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:54,476 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:54,476 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:54,476 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:54,477 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:54,477 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:54,477 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:54,478 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:54,478 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:54,478 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:54,478 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,479 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,479 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:54,480 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,480 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,480 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:54,481 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:54,481 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:54,481 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:54,482 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,482 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:54,482 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,482 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,482 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,482 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:01:54,483 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:54,484 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:54,484 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325744484 2023-02-13 22:01:54,485 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:54,485 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,487 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:54,487 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:54,487 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:54,488 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325774488 2023-02-13 22:01:54,488 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325834488 2023-02-13 22:01:54,488 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:01:54,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN}] 2023-02-13 22:01:54,488 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:54,490 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN 2023-02-13 22:01:54,491 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=fe45e04d204e94f79d1eca4598523369, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:54,491 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325714491"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325714491"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325714491"}]},"ts":"1676325714491"} 2023-02-13 22:01:54,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:01:54,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing fe45e04d204e94f79d1eca4598523369, disabling compactions & flushes 2023-02-13 22:01:54,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:54,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:54,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. after waiting 0 ms 2023-02-13 22:01:54,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:54,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing fe45e04d204e94f79d1eca4598523369 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 22:01:54,664 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/b75609aabc64444e8307c6838e5446e2 2023-02-13 22:01:54,685 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/1f18f1efc06e4748873c0469ce008ef8 2023-02-13 22:01:54,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf1/b75609aabc64444e8307c6838e5446e2 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2 2023-02-13 22:01:54,692 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:01:54,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/.tmp/cf2/1f18f1efc06e4748873c0469ce008ef8 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8 2023-02-13 22:01:54,696 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:01:54,696 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for fe45e04d204e94f79d1eca4598523369 in 51ms, sequenceid=82, compaction requested=true 2023-02-13 22:01:54,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 22:01:54,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:01:54,701 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. 2023-02-13 22:01:54,701 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for fe45e04d204e94f79d1eca4598523369: Waiting for close lock at 1676325714644Running coprocessor pre-close hooks at 1676325714644Disabling compacts and flushes for region at 1676325714644Disabling writes for close at 1676325714644Obtaining lock to block concurrent updates at 1676325714645 (+1 ms)Preparing flush snapshotting stores in fe45e04d204e94f79d1eca4598523369 at 1676325714645Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676325714645Flushing stores of testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. at 1676325714646 (+1 ms)Flushing fe45e04d204e94f79d1eca4598523369/cf1: creating writer at 1676325714648 (+2 ms)Flushing fe45e04d204e94f79d1eca4598523369/cf1: appending metadata at 1676325714653 (+5 ms)Flushing fe45e04d204e94f79d1eca4598523369/cf1: closing flushed file at 1676325714653Flushing fe45e04d204e94f79d1eca4598523369/cf2: creating writer at 1676325714671 (+18 ms)Flushing fe45e04d204e94f79d1eca4598523369/cf2: appending metadata at 1676325714674 (+3 ms)Flushing fe45e04d204e94f79d1eca4598523369/cf2: closing flushed file at 1676325714674Flushing fe45e04d204e94f79d1eca4598523369/cf1: reopening flushed file at 1676325714689 (+15 ms)Flushing fe45e04d204e94f79d1eca4598523369/cf2: reopening flushed file at 1676325714693 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for fe45e04d204e94f79d1eca4598523369 in 51ms, sequenceid=82, compaction requested=true at 1676325714696 (+3 ms)Writing region close event to WAL at 1676325714697 (+1 ms)Running coprocessor post-close hooks at 1676325714700 (+3 ms)Closed at 1676325714701 (+1 ms) 2023-02-13 22:01:54,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,703 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=fe45e04d204e94f79d1eca4598523369, regionState=CLOSED 2023-02-13 22:01:54,703 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325714703"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325714703"}]},"ts":"1676325714703"} 2023-02-13 22:01:54,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-13 22:01:54,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 in 212 msec 2023-02-13 22:01:54,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-13 22:01:54,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN in 218 msec 2023-02-13 22:01:54,709 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:54,709 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:54,709 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:54,738 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,738 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:54,738 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:54,738 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:54,738 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:01:54,739 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:54,739 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:54,739 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:54,739 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:54,739 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:01:54,742 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 22:01:54,742 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:01:54,742 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:01:54,742 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:54,743 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:54,743 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:54,743 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:54,743 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:54,744 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:54,744 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:54,744 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:54,744 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:54,744 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:54,745 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:54,745 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:54,745 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:54,745 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:54,746 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:54,746 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:54,746 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:54,747 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:54,747 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:54,747 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:54,747 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:54,748 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:54,748 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:54,748 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:54,749 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:54,749 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,749 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,750 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:54,750 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,750 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:54,750 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:54,750 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:54,750 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:54,751 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:54,751 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,751 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN 2023-02-13 22:01:54,751 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:54,751 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:54,751 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,751 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,751 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,751 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:54,751 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:54,751 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-13 22:01:54,752 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:54,752 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:54,753 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325744753 2023-02-13 22:01:54,753 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:54,754 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:54,756 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:54,756 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:54,756 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:54,756 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325774756 2023-02-13 22:01:54,756 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325834756 2023-02-13 22:01:54,756 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:01:54,759 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:54,769 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=fe45e04d204e94f79d1eca4598523369, threads=14 2023-02-13 22:01:54,769 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,769 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,769 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,769 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,769 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,769 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,769 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,769 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,770 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,770 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,770 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,772 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,772 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,773 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,865 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,875 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,892 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 22:01:54,893 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,893 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,893 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,894 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,903 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,913 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,915 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,918 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,918 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:54,929 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,181 ERROR [BP-109365581-148.251.75.209-1676325611364 heartbeating to localhost.localdomain/127.0.0.1:37463] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-109365581-148.251.75.209-1676325611364 (Datanode Uuid 643297d9-b5a4-472e-97c3-eb890073d22e) service to localhost.localdomain/127.0.0.1:37463 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 22:01:55,323 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,330 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,330 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region fe45e04d204e94f79d1eca4598523369 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 22:01:55,379 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:55,379 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:55,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:55,379 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:55,507 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:55,507 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:55,507 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:55,507 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:55,508 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:01:55,508 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:55,508 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:55,509 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:55,509 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:55,510 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:01:55,514 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 22:01:55,514 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:01:55,514 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:01:55,515 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:55,515 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:55,515 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:55,515 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:55,515 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:55,515 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:55,516 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:55,516 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:55,516 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:55,516 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:55,516 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:55,516 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:55,516 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:55,517 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:55,517 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:55,517 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:55,517 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:55,517 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:55,517 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:55,518 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:55,518 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:55,518 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:55,518 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:55,518 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:55,518 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:55,518 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:55,519 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:55,519 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:55,519 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:55,519 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:55,519 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:55,519 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:55,520 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:55,520 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:55,520 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:55,520 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN 2023-02-13 22:01:55,520 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:55,520 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:55,520 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:55,520 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:55,520 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:55,521 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:55,521 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:55,521 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:01:55,521 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:55,522 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:55,522 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325745522 2023-02-13 22:01:55,524 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:55,524 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:55,526 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:55,526 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:55,526 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:55,526 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325775526 2023-02-13 22:01:55,526 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325835526 2023-02-13 22:01:55,526 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:01:55,534 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=fe45e04d204e94f79d1eca4598523369, threads=14 2023-02-13 22:01:55,534 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,534 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,534 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,535 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,535 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,536 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,536 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,536 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,537 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,537 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,537 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,539 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,539 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,539 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,599 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,600 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,607 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,608 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,610 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,611 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,614 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,615 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,615 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:55,996 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:56,005 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:56,012 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:56,014 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:56,014 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947 for region: fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:56,015 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region fe45e04d204e94f79d1eca4598523369 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 22:01:56,081 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:56,089 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:01:56,092 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:01:56,093 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:56,093 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:56,093 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:56,277 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,277 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:56,277 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:56,278 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:56,278 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:01:56,278 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:56,279 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:56,279 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:56,279 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:56,281 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:01:56,284 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 22:01:56,284 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:01:56,284 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:01:56,284 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:56,285 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:56,285 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:56,285 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:56,286 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:56,286 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:56,286 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:56,287 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:56,287 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:56,287 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:56,288 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:56,288 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:56,288 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:56,289 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:56,289 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:56,289 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:56,289 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:56,290 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:56,290 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:56,290 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:56,291 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:56,291 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:56,291 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:56,292 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:56,292 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:56,292 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,293 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,293 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:56,293 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,294 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,294 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:56,294 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:56,294 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:56,295 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:56,295 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,295 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN 2023-02-13 22:01:56,295 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:56,295 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:56,295 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,295 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,295 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,295 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:56,296 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:56,296 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 14 msec 2023-02-13 22:01:56,296 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:56,297 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:56,297 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325746297 2023-02-13 22:01:56,298 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:56,298 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,300 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:56,301 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:56,301 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:56,301 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325776301 2023-02-13 22:01:56,301 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325836301 2023-02-13 22:01:56,301 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:01:56,303 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:56,305 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:56,305 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:56,305 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:56,551 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,551 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:56,552 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:56,552 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:56,552 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:01:56,553 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:56,553 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:56,553 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:56,553 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:56,555 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:01:56,557 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 22:01:56,557 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:01:56,557 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:01:56,557 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:56,558 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:56,558 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:56,558 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:56,559 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:56,559 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:56,559 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:56,559 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:56,560 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:56,560 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:56,560 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:56,560 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:56,561 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:56,561 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:56,561 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:56,562 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:56,562 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:56,562 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:56,562 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:56,563 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:56,563 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:56,563 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:56,564 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:56,564 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:56,564 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:56,564 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,565 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,565 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:56,565 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,566 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,566 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:56,566 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:56,566 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:56,567 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:56,567 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,567 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN 2023-02-13 22:01:56,567 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:56,567 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:56,567 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,567 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,567 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,567 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:56,567 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:56,567 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-13 22:01:56,568 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:56,569 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:56,569 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325746569 2023-02-13 22:01:56,570 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,570 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:56,573 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:56,573 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:56,573 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:56,573 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325776573 2023-02-13 22:01:56,573 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325836573 2023-02-13 22:01:56,573 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:01:56,573 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:56,575 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325716575"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676325716575"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676325716575"}]},"ts":"1676325716575"} 2023-02-13 22:01:56,576 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325716575"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325716575"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325716575"}]},"ts":"1676325716575"} 2023-02-13 22:01:56,576 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325716575"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325716575"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325716575"}]},"ts":"1676325716575"} 2023-02-13 22:01:56,577 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:56,577 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:56,577 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:56,823 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,824 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:56,824 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:56,824 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:56,824 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:01:56,825 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:56,825 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:56,825 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:56,825 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:56,827 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:01:56,832 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 22:01:56,833 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:01:56,833 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:01:56,834 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:56,835 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:56,835 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:56,836 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:56,836 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:56,836 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:56,836 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:56,837 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:56,837 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:56,837 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:56,837 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:56,838 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:56,838 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:56,839 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:56,839 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:56,839 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:56,839 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:56,839 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:56,840 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:56,840 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:56,840 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:56,840 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:56,840 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:56,841 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:56,841 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:56,841 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,841 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,842 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:56,842 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,842 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:56,842 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:56,842 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:56,843 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:56,843 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:56,843 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,843 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN 2023-02-13 22:01:56,843 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:56,843 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:56,844 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,844 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,844 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,844 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:56,844 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:56,844 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 17 msec 2023-02-13 22:01:56,845 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:56,845 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:56,846 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325746846 2023-02-13 22:01:56,847 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,847 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:56,850 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:56,850 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:56,850 INFO [Listener at localhost.localdomain/42295] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:56,850 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=5108c25fa5b38236fddb2d76fe4159a5, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 22:01:56,850 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=ef0fd3f6beca364dd7ba6dfbc83ab92c, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 22:01:56,850 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:56,850 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325776850 2023-02-13 22:01:56,850 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325836850 2023-02-13 22:01:56,850 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:01:56,851 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676325716851"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676325716851"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676325716851"}]},"ts":"1676325716851"} 2023-02-13 22:01:56,851 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325716851"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325716851"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325716851"}]},"ts":"1676325716851"} 2023-02-13 22:01:56,851 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325716851"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325716851"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325716851"}]},"ts":"1676325716851"} 2023-02-13 22:01:56,852 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:56,854 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:56,854 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:56,854 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:56,854 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:57,101 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,101 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:57,101 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:57,101 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:57,103 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:01:57,103 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:57,104 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:57,104 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:57,104 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:57,105 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:01:57,108 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-13 22:01:57,109 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:01:57,109 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:01:57,109 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:57,110 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:57,110 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:57,111 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:57,111 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:57,112 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:57,112 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:57,112 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:57,113 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:57,113 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:57,113 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:57,113 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:57,114 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:57,114 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:57,114 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:57,114 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:57,114 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:57,114 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:57,114 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:57,115 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:57,115 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:57,115 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:57,115 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:57,115 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:57,115 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:57,115 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:57,115 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:57,116 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:57,116 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:57,116 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:57,116 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:57,116 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:57,116 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:57,116 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:57,116 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,116 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN 2023-02-13 22:01:57,117 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:57,117 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:57,117 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,117 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,117 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,117 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:57,117 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:57,117 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-13 22:01:57,118 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:57,118 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:57,118 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325747118 2023-02-13 22:01:57,119 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,120 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:57,122 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:57,122 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:57,122 INFO [Listener at localhost.localdomain/42295] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:57,122 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=5108c25fa5b38236fddb2d76fe4159a5, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 22:01:57,122 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=ef0fd3f6beca364dd7ba6dfbc83ab92c, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 22:01:57,122 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:57,122 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325777122 2023-02-13 22:01:57,122 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325837122 2023-02-13 22:01:57,122 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 22:01:57,122 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:57,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5108c25fa5b38236fddb2d76fe4159a5, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ef0fd3f6beca364dd7ba6dfbc83ab92c, ASSIGN}] 2023-02-13 22:01:57,124 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:57,124 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:57,124 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:57,373 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,373 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:57,373 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:57,373 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:57,374 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:01:57,374 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:57,374 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:57,374 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:57,375 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:57,376 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 22:01:57,380 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-13 22:01:57,381 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:01:57,381 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:01:57,381 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:57,382 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:57,383 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:57,383 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:57,383 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:57,383 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:57,383 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:57,383 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:57,383 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:57,384 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:57,384 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:57,384 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:57,384 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:57,384 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:57,384 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:57,384 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:57,384 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:57,384 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:57,384 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:57,385 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:57,385 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:57,385 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:57,385 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:57,385 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:57,385 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:57,385 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:57,385 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:57,385 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:57,386 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:57,386 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:57,386 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:57,386 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:57,386 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:57,386 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:57,386 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,387 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN 2023-02-13 22:01:57,387 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:57,387 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:57,387 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,387 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,387 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,387 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:57,387 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:57,387 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:01:57,388 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:57,388 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:57,389 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325747389 2023-02-13 22:01:57,389 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:57,390 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,392 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:57,392 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:57,392 INFO [Listener at localhost.localdomain/42295] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:57,393 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=5108c25fa5b38236fddb2d76fe4159a5, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 22:01:57,393 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=ef0fd3f6beca364dd7ba6dfbc83ab92c, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 22:01:57,393 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:57,393 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325777393 2023-02-13 22:01:57,393 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325837393 2023-02-13 22:01:57,393 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 22:01:57,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5108c25fa5b38236fddb2d76fe4159a5, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ef0fd3f6beca364dd7ba6dfbc83ab92c, ASSIGN}] 2023-02-13 22:01:57,393 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:57,395 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ef0fd3f6beca364dd7ba6dfbc83ab92c, ASSIGN 2023-02-13 22:01:57,396 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ef0fd3f6beca364dd7ba6dfbc83ab92c, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,42047,1676325615467; forceNewPlan=false, retain=false 2023-02-13 22:01:57,396 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5108c25fa5b38236fddb2d76fe4159a5, ASSIGN 2023-02-13 22:01:57,396 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5108c25fa5b38236fddb2d76fe4159a5, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,42047,1676325615467; forceNewPlan=false, retain=false 2023-02-13 22:01:57,546 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 22:01:57,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=5108c25fa5b38236fddb2d76fe4159a5, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:57,549 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325717548"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325717548"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325717548"}]},"ts":"1676325717548"} 2023-02-13 22:01:57,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 5108c25fa5b38236fddb2d76fe4159a5, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:01:57,557 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=ef0fd3f6beca364dd7ba6dfbc83ab92c, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:57,557 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325717557"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325717557"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325717557"}]},"ts":"1676325717557"} 2023-02-13 22:01:57,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure ef0fd3f6beca364dd7ba6dfbc83ab92c, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:01:57,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:57,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ef0fd3f6beca364dd7ba6dfbc83ab92c, NAME => 'testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 22:01:57,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:01:57,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:01:57,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,714 INFO [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,716 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1 2023-02-13 22:01:57,716 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1 2023-02-13 22:01:57,717 INFO [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ef0fd3f6beca364dd7ba6dfbc83ab92c columnFamilyName cf1 2023-02-13 22:01:57,734 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701-top 2023-02-13 22:01:57,739 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d-top 2023-02-13 22:01:57,743 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0-top 2023-02-13 22:01:57,747 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1-top 2023-02-13 22:01:57,750 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2-top 2023-02-13 22:01:57,754 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c-top 2023-02-13 22:01:57,758 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14-top 2023-02-13 22:01:57,758 INFO [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(310): Store=ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:57,758 INFO [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,759 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2 2023-02-13 22:01:57,759 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2 2023-02-13 22:01:57,759 INFO [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ef0fd3f6beca364dd7ba6dfbc83ab92c columnFamilyName cf2 2023-02-13 22:01:57,771 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30-top 2023-02-13 22:01:57,774 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8-top 2023-02-13 22:01:57,778 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520-top 2023-02-13 22:01:57,781 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc-top 2023-02-13 22:01:57,784 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9-top 2023-02-13 22:01:57,788 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7-top 2023-02-13 22:01:57,791 DEBUG [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947-top 2023-02-13 22:01:57,791 INFO [StoreOpener-ef0fd3f6beca364dd7ba6dfbc83ab92c-1] regionserver.HStore(310): Store=ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:57,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:01:57,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened ef0fd3f6beca364dd7ba6dfbc83ab92c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71853058, jitterRate=0.07069399952888489}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:01:57,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for ef0fd3f6beca364dd7ba6dfbc83ab92c: Running coprocessor pre-open hook at 1676325717711Writing region info on filesystem at 1676325717711Initializing all the Stores at 1676325717713 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325717713Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325717713Cleaning up temporary data from old regions at 1676325717793 (+80 ms)Cleaning up detritus from prior splits at 1676325717793Running coprocessor post-open hooks at 1676325717795 (+2 ms)Region opened successfully at 1676325717795 2023-02-13 22:01:57,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c., pid=118, masterSystemTime=1676325717704 2023-02-13 22:01:57,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:01:57,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:01:57,798 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:01:57,799 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:01:57,799 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:57,799 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:57,799 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1912): ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1 is initiating minor compaction (all files) 2023-02-13 22:01:57,799 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1912): ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2 is initiating minor compaction (all files) 2023-02-13 22:01:57,799 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1 in testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:57,799 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegion(2258): Starting compaction of ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2 in testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:57,799 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8-top] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/.tmp, totalSize=34.9 K 2023-02-13 22:01:57,799 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2-top] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/.tmp, totalSize=34.9 K 2023-02-13 22:01:57,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:57,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:57,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:57,800 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676325712233 2023-02-13 22:01:57,800 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676325712233 2023-02-13 22:01:57,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5108c25fa5b38236fddb2d76fe4159a5, NAME => 'testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 22:01:57,800 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=ef0fd3f6beca364dd7ba6dfbc83ab92c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:57,800 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676325712401 2023-02-13 22:01:57,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:01:57,800 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325717800"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325717800"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325717800"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325717800"}]},"ts":"1676325717800"} 2023-02-13 22:01:57,800 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting 21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676325712401 2023-02-13 22:01:57,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:57,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:01:57,801 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676325712610 2023-02-13 22:01:57,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:57,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:57,801 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676325712610 2023-02-13 22:01:57,801 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676325712811 2023-02-13 22:01:57,801 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676325712992 2023-02-13 22:01:57,801 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting 10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676325712811 2023-02-13 22:01:57,801 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676325713196 2023-02-13 22:01:57,802 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting 27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676325712992 2023-02-13 22:01:57,802 INFO [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:57,802 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting 240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676325713196 2023-02-13 22:01:57,802 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676325713411 2023-02-13 22:01:57,802 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting 1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676325713411 2023-02-13 22:01:57,807 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1 2023-02-13 22:01:57,807 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1 2023-02-13 22:01:57,808 INFO [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5108c25fa5b38236fddb2d76fe4159a5 columnFamilyName cf1 2023-02-13 22:01:57,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-13 22:01:57,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure ef0fd3f6beca364dd7ba6dfbc83ab92c, server=jenkins-hbase20.apache.org,42047,1676325615467 in 242 msec 2023-02-13 22:01:57,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ef0fd3f6beca364dd7ba6dfbc83ab92c, ASSIGN in 415 msec 2023-02-13 22:01:57,819 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] throttle.PressureAwareThroughputController(145): ef0fd3f6beca364dd7ba6dfbc83ab92c#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:01:57,820 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] throttle.PressureAwareThroughputController(145): ef0fd3f6beca364dd7ba6dfbc83ab92c#cf2#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:01:57,824 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701-bottom 2023-02-13 22:01:57,827 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d-bottom 2023-02-13 22:01:57,837 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0-bottom 2023-02-13 22:01:57,842 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/.tmp/cf1/e6cb1a0f4612423b97f6b249d3dbf1b3 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/e6cb1a0f4612423b97f6b249d3dbf1b3 2023-02-13 22:01:57,848 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1-bottom 2023-02-13 22:01:57,851 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2-bottom 2023-02-13 22:01:57,854 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1 of ef0fd3f6beca364dd7ba6dfbc83ab92c into e6cb1a0f4612423b97f6b249d3dbf1b3(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:01:57,854 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for ef0fd3f6beca364dd7ba6dfbc83ab92c: 2023-02-13 22:01:57,854 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c., storeName=ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1, priority=9, startTime=1676325717796; duration=0sec 2023-02-13 22:01:57,854 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:01:57,855 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c-bottom 2023-02-13 22:01:57,859 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/.tmp/cf2/0c7469a4d97841d39e134add2931acaf as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/0c7469a4d97841d39e134add2931acaf 2023-02-13 22:01:57,859 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14-bottom 2023-02-13 22:01:57,859 INFO [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(310): Store=5108c25fa5b38236fddb2d76fe4159a5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:57,859 INFO [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:57,860 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2 2023-02-13 22:01:57,860 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2 2023-02-13 22:01:57,861 INFO [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5108c25fa5b38236fddb2d76fe4159a5 columnFamilyName cf2 2023-02-13 22:01:57,863 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2 of ef0fd3f6beca364dd7ba6dfbc83ab92c into 0c7469a4d97841d39e134add2931acaf(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:01:57,863 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for ef0fd3f6beca364dd7ba6dfbc83ab92c: 2023-02-13 22:01:57,863 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c., storeName=ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2, priority=9, startTime=1676325717798; duration=0sec 2023-02-13 22:01:57,863 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:01:57,873 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30-bottom 2023-02-13 22:01:57,876 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8-bottom 2023-02-13 22:01:57,878 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520-bottom 2023-02-13 22:01:57,882 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc-bottom 2023-02-13 22:01:57,885 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9-bottom 2023-02-13 22:01:57,888 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7-bottom 2023-02-13 22:01:57,893 DEBUG [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947-bottom 2023-02-13 22:01:57,893 INFO [StoreOpener-5108c25fa5b38236fddb2d76fe4159a5-1] regionserver.HStore(310): Store=5108c25fa5b38236fddb2d76fe4159a5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:57,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:57,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:57,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:01:57,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:57,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 5108c25fa5b38236fddb2d76fe4159a5; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60778028, jitterRate=-0.09433680772781372}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:01:57,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 5108c25fa5b38236fddb2d76fe4159a5: Running coprocessor pre-open hook at 1676325717801Writing region info on filesystem at 1676325717801Initializing all the Stores at 1676325717801Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325717801Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325717802 (+1 ms)Cleaning up temporary data from old regions at 1676325717895 (+93 ms)Cleaning up detritus from prior splits at 1676325717895Running coprocessor post-open hooks at 1676325717897 (+2 ms)Region opened successfully at 1676325717897 2023-02-13 22:01:57,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5., pid=117, masterSystemTime=1676325717704 2023-02-13 22:01:57,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:01:57,899 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:01:57,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:01:57,899 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:01:57,899 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:57,899 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1912): 5108c25fa5b38236fddb2d76fe4159a5/cf1 is initiating minor compaction (all files) 2023-02-13 22:01:57,899 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:57,899 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1912): 5108c25fa5b38236fddb2d76fe4159a5/cf2 is initiating minor compaction (all files) 2023-02-13 22:01:57,899 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5108c25fa5b38236fddb2d76fe4159a5/cf1 in testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:57,899 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5108c25fa5b38236fddb2d76fe4159a5/cf2 in testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:57,900 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2-bottom] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/.tmp, totalSize=34.9 K 2023-02-13 22:01:57,900 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8-bottom] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/.tmp, totalSize=34.9 K 2023-02-13 22:01:57,900 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325712233 2023-02-13 22:01:57,900 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325712233 2023-02-13 22:01:57,901 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325712401 2023-02-13 22:01:57,901 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting 21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325712401 2023-02-13 22:01:57,901 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325712610 2023-02-13 22:01:57,901 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325712610 2023-02-13 22:01:57,901 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325712811 2023-02-13 22:01:57,902 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=5108c25fa5b38236fddb2d76fe4159a5, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:57,902 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325717902"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325717902"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325717902"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325717902"}]},"ts":"1676325717902"} 2023-02-13 22:01:57,902 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting 6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325712992 2023-02-13 22:01:57,902 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting 10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325712811 2023-02-13 22:01:57,902 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325713196 2023-02-13 22:01:57,902 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting 27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325712992 2023-02-13 22:01:57,903 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] compactions.Compactor(207): Compacting b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325713411 2023-02-13 22:01:57,903 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting 240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325713196 2023-02-13 22:01:57,904 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] compactions.Compactor(207): Compacting 1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325713411 2023-02-13 22:01:57,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:57,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:57,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-13 22:01:57,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 5108c25fa5b38236fddb2d76fe4159a5, server=jenkins-hbase20.apache.org,42047,1676325615467 in 352 msec 2023-02-13 22:01:57,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-13 22:01:57,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5108c25fa5b38236fddb2d76fe4159a5, ASSIGN in 514 msec 2023-02-13 22:01:57,908 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 22:01:57,908 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:57,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:01:57,908 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 22:01:57,913 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5108c25fa5b38236fddb2d76fe4159a5#cf1#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:01:57,913 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] throttle.PressureAwareThroughputController(145): 5108c25fa5b38236fddb2d76fe4159a5#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:01:57,927 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/.tmp/cf1/e647628404374b638b7e2af1784e5911 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/e647628404374b638b7e2af1784e5911 2023-02-13 22:01:57,929 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/.tmp/cf2/73dfc79b9fb84aa7acd9bce3ce39c6ca as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/73dfc79b9fb84aa7acd9bce3ce39c6ca 2023-02-13 22:01:57,931 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5108c25fa5b38236fddb2d76fe4159a5/cf1 of 5108c25fa5b38236fddb2d76fe4159a5 into e647628404374b638b7e2af1784e5911(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:01:57,931 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5108c25fa5b38236fddb2d76fe4159a5: 2023-02-13 22:01:57,931 INFO [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5., storeName=5108c25fa5b38236fddb2d76fe4159a5/cf1, priority=9, startTime=1676325717898; duration=0sec 2023-02-13 22:01:57,931 DEBUG [RS:2;jenkins-hbase20:42047-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:01:57,932 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5108c25fa5b38236fddb2d76fe4159a5/cf2 of 5108c25fa5b38236fddb2d76fe4159a5 into 73dfc79b9fb84aa7acd9bce3ce39c6ca(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:01:57,932 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5108c25fa5b38236fddb2d76fe4159a5: 2023-02-13 22:01:57,932 INFO [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5., storeName=5108c25fa5b38236fddb2d76fe4159a5/cf2, priority=9, startTime=1676325717899; duration=0sec 2023-02-13 22:01:57,932 DEBUG [RS:2;jenkins-hbase20:42047-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:01:58,143 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:58,143 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 22:01:58,144 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:01:58,144 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:01:58,144 ERROR [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:01:58,144 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 22:01:58,144 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 22:01:58,144 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 22:01:58,144 INFO [Listener at localhost.localdomain/42295] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:01:58,145 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:01:58,146 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-13 22:01:58,147 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 22:01:58,147 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 22:01:58,147 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 23] 2023-02-13 22:01:58,147 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-02-13 22:01:58,147 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 21] 2023-02-13 22:01:58,147 DEBUG [Listener at localhost.localdomain/42295] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-02-13 22:01:58,147 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:01:58,147 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:01:58,148 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:01:58,148 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:01:58,148 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:58,148 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=091ee0643bbb09cdba3c6e8d5b8e5835, daughterA=d4dc0951c7044c39c81d72b688a3b019, daughterB=a8a1dfec1256014dc4fdfc9c8ef9862d 2023-02-13 22:01:58,148 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:58,149 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 22:01:58,149 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:58,149 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4fff278c72d1992f129cd33a6d722426, daughterA=b458c667c868fe528fcdd3363340873b, daughterB=245a056601be8d7d102a0dc6a730bbad 2023-02-13 22:01:58,149 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:58,149 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 22:01:58,149 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:58,150 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=386c949cc4d06cf801b328070dc3f9d1, daughterA=eff524ec09e63cfead417cc59f09f92c, daughterB=a866d0f6ae566c2e8c561a5af50aa06e 2023-02-13 22:01:58,150 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=386c949cc4d06cf801b328070dc3f9d1, ASSIGN 2023-02-13 22:01:58,150 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:58,150 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 22:01:58,150 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:58,150 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=40b25f110a291fb9097d057d629f605c, daughterA=2dca75ce4d92efcc1c669fe0165d9507, daughterB=61a4873d60328da5b81ad52cecec03ce 2023-02-13 22:01:58,151 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:58,151 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 22:01:58,151 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:58,151 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=b54fd364e5081ef6e29844259f5ed76e, daughterA=1405218340a9d9a499f287b55eca8ccc, daughterB=db1b569d476c9340ddb2430f438d297d 2023-02-13 22:01:58,151 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:58,151 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 22:01:58,152 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:58,152 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:58,152 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRollbackForSplitTableRegionWithReplica, region=08a32e58e596f7f48d8374c3c2c4f4b3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=faa5a059f8b3120d55e00b3ba9ebb6b9, daughterA=c31c3f5ea2ddace8f1314b31357c4229, daughterB=564432147fecd5909dbda2427be8d684 2023-02-13 22:01:58,152 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:58,152 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 22:01:58,153 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:58,153 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:58,153 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 22:01:58,153 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:58,154 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:58,154 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN 2023-02-13 22:01:58,154 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:58,154 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5108c25fa5b38236fddb2d76fe4159a5, ASSIGN 2023-02-13 22:01:58,154 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ef0fd3f6beca364dd7ba6dfbc83ab92c, ASSIGN 2023-02-13 22:01:58,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 5108c25fa5b38236fddb2d76fe4159a5, server=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:58,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure ef0fd3f6beca364dd7ba6dfbc83ab92c, server=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:58,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c held the lock before restarting, call acquireLock to restore it. 2023-02-13 22:01:58,155 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:58,155 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:58,155 INFO [Listener at localhost.localdomain/42295] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:58,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:58,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure fe45e04d204e94f79d1eca4598523369, server=jenkins-hbase20.apache.org,42047,1676325615467 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:58,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5108c25fa5b38236fddb2d76fe4159a5, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:58,155 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ef0fd3f6beca364dd7ba6dfbc83ab92c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:58,156 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 5108c25fa5b38236fddb2d76fe4159a5, server=jenkins-hbase20.apache.org,42047,1676325615467 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:58,156 DEBUG [Listener at localhost.localdomain/42295] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure ef0fd3f6beca364dd7ba6dfbc83ab92c, server=jenkins-hbase20.apache.org,42047,1676325615467 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 22:01:58,156 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 22:01:58,157 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:01:58,158 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,42047,1676325615467, table=hbase:meta, region=1588230740 2023-02-13 22:01:58,158 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325748158 2023-02-13 22:01:58,159 DEBUG [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:01:58,159 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:58,162 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=09ae03d8301e349bcc2ee47a4f5f6b0a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:58,162 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=fe45e04d204e94f79d1eca4598523369, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=2 2023-02-13 22:01:58,162 INFO [Listener at localhost.localdomain/42295] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,42047,1676325615467, table=testRecoveryAndDoubleExecution, region=fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,163 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=5108c25fa5b38236fddb2d76fe4159a5, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=86 2023-02-13 22:01:58,163 DEBUG [Listener at localhost.localdomain/42295] assignment.RegionStateStore(147): Load hbase:meta entry region=ef0fd3f6beca364dd7ba6dfbc83ab92c, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,42047,1676325615467, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467, openSeqNum=86 2023-02-13 22:01:58,163 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:01:58,163 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325778163 2023-02-13 22:01:58,163 INFO [Listener at localhost.localdomain/42295] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325838163 2023-02-13 22:01:58,163 INFO [Listener at localhost.localdomain/42295] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 22:01:58,163 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 22:01:58,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=fe45e04d204e94f79d1eca4598523369, daughterA=5108c25fa5b38236fddb2d76fe4159a5, daughterB=ef0fd3f6beca364dd7ba6dfbc83ab92c in 4.4850 sec 2023-02-13 22:01:58,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42047] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 148.251.75.209:45782 deadline: 1676325778415, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369. is not online on jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:58,534 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 22:01:58,542 INFO [Listener at localhost.localdomain/42295] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 22:01:58,544 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 22:01:58,555 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:01:58,557 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-13 22:01:58,557 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testRecoveryAndDoubleExecution 2023-02-13 22:01:58,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:58,560 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325718560"}]},"ts":"1676325718560"} 2023-02-13 22:01:58,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 22:01:58,561 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 22:01:58,561 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-13 22:01:58,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5108c25fa5b38236fddb2d76fe4159a5, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ef0fd3f6beca364dd7ba6dfbc83ab92c, UNASSIGN}] 2023-02-13 22:01:58,562 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ef0fd3f6beca364dd7ba6dfbc83ab92c, UNASSIGN 2023-02-13 22:01:58,563 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=ef0fd3f6beca364dd7ba6dfbc83ab92c, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:58,563 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325718563"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325718563"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325718563"}]},"ts":"1676325718563"} 2023-02-13 22:01:58,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure ef0fd3f6beca364dd7ba6dfbc83ab92c, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:01:58,565 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5108c25fa5b38236fddb2d76fe4159a5, UNASSIGN 2023-02-13 22:01:58,565 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=5108c25fa5b38236fddb2d76fe4159a5, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:58,565 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325718565"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325718565"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325718565"}]},"ts":"1676325718565"} 2023-02-13 22:01:58,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 5108c25fa5b38236fddb2d76fe4159a5, server=jenkins-hbase20.apache.org,42047,1676325615467}] 2023-02-13 22:01:58,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 22:01:58,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:58,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 5108c25fa5b38236fddb2d76fe4159a5, disabling compactions & flushes 2023-02-13 22:01:58,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:58,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:58,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. after waiting 0 ms 2023-02-13 22:01:58,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:58,718 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2-bottom] to archive 2023-02-13 22:01:58,721 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:01:58,726 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,728 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,730 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,732 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,733 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,735 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,736 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,738 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8-bottom] to archive 2023-02-13 22:01:58,738 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:01:58,740 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,742 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,743 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,744 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,746 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,747 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,748 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,752 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 22:01:58,752 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:01:58,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5. 2023-02-13 22:01:58,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 5108c25fa5b38236fddb2d76fe4159a5: Waiting for close lock at 1676325718716Running coprocessor pre-close hooks at 1676325718716Disabling compacts and flushes for region at 1676325718716Disabling writes for close at 1676325718717 (+1 ms)Writing region close event to WAL at 1676325718749 (+32 ms)Running coprocessor post-close hooks at 1676325718752 (+3 ms)Closed at 1676325718753 (+1 ms) 2023-02-13 22:01:58,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:58,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:58,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing ef0fd3f6beca364dd7ba6dfbc83ab92c, disabling compactions & flushes 2023-02-13 22:01:58,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:58,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:58,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. after waiting 0 ms 2023-02-13 22:01:58,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:58,755 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=5108c25fa5b38236fddb2d76fe4159a5, regionState=CLOSED 2023-02-13 22:01:58,755 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325718755"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325718755"}]},"ts":"1676325718755"} 2023-02-13 22:01:58,756 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2-top] to archive 2023-02-13 22:01:58,757 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:01:58,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-13 22:01:58,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 5108c25fa5b38236fddb2d76fe4159a5, server=jenkins-hbase20.apache.org,42047,1676325615467 in 191 msec 2023-02-13 22:01:58,759 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5108c25fa5b38236fddb2d76fe4159a5, UNASSIGN in 196 msec 2023-02-13 22:01:58,761 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,762 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,763 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,765 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,766 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,767 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,770 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8-top] to archive 2023-02-13 22:01:58,770 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:01:58,772 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,773 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,775 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,776 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,778 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,780 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,781 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 22:01:58,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:01:58,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c. 2023-02-13 22:01:58,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for ef0fd3f6beca364dd7ba6dfbc83ab92c: Waiting for close lock at 1676325718755Running coprocessor pre-close hooks at 1676325718755Disabling compacts and flushes for region at 1676325718755Disabling writes for close at 1676325718755Writing region close event to WAL at 1676325718782 (+27 ms)Running coprocessor post-close hooks at 1676325718786 (+4 ms)Closed at 1676325718786 2023-02-13 22:01:58,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:58,790 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=ef0fd3f6beca364dd7ba6dfbc83ab92c, regionState=CLOSED 2023-02-13 22:01:58,790 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676325718790"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325718790"}]},"ts":"1676325718790"} 2023-02-13 22:01:58,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-13 22:01:58,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure ef0fd3f6beca364dd7ba6dfbc83ab92c, server=jenkins-hbase20.apache.org,42047,1676325615467 in 228 msec 2023-02-13 22:01:58,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-02-13 22:01:58,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ef0fd3f6beca364dd7ba6dfbc83ab92c, UNASSIGN in 232 msec 2023-02-13 22:01:58,797 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325718797"}]},"ts":"1676325718797"} 2023-02-13 22:01:58,798 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 22:01:58,800 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-13 22:01:58,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 244 msec 2023-02-13 22:01:58,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 22:01:58,864 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-13 22:01:58,864 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testRecoveryAndDoubleExecution 2023-02-13 22:01:58,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:58,867 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:58,868 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:58,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 22:01:58,873 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,873 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:58,873 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:58,876 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/recovered.edits] 2023-02-13 22:01:58,876 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/recovered.edits] 2023-02-13 22:01:58,876 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/recovered.edits] 2023-02-13 22:01:58,879 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/e6cb1a0f4612423b97f6b249d3dbf1b3 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/e6cb1a0f4612423b97f6b249d3dbf1b3 2023-02-13 22:01:58,880 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/.regioninfo to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/.regioninfo 2023-02-13 22:01:58,881 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/5d52eeef35a94f32bb7b454c4e087701 2023-02-13 22:01:58,882 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/6884cbf58a2446e9a50a05fb5832f51d 2023-02-13 22:01:58,883 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/0c7469a4d97841d39e134add2931acaf to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/0c7469a4d97841d39e134add2931acaf 2023-02-13 22:01:58,884 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,884 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/80bb277e72704b8c99cdf87e3fd889e0 2023-02-13 22:01:58,885 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,885 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/88c5fddce6074a2787a34282e42fe7f1 2023-02-13 22:01:58,886 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/.regioninfo to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/.regioninfo 2023-02-13 22:01:58,886 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,886 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b75609aabc64444e8307c6838e5446e2 2023-02-13 22:01:58,887 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,887 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/b8905f0ed6fa408bb58db5cfc3672b9c 2023-02-13 22:01:58,888 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/5d52eeef35a94f32bb7b454c4e087701.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,888 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,889 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf1/efa49219a1404a4f8f21adbde804fa14 2023-02-13 22:01:58,891 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/6884cbf58a2446e9a50a05fb5832f51d.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,891 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,892 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/80bb277e72704b8c99cdf87e3fd889e0.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,892 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,893 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/10a3bfd393ee45b7b9a65d3bb7040b30 2023-02-13 22:01:58,893 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/88c5fddce6074a2787a34282e42fe7f1.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,894 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/1f18f1efc06e4748873c0469ce008ef8 2023-02-13 22:01:58,894 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b75609aabc64444e8307c6838e5446e2.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,895 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/21da36534a1140dea813aacdae9c8520 2023-02-13 22:01:58,895 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,896 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/b8905f0ed6fa408bb58db5cfc3672b9c.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,896 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/240f909ea4d54bda9771214216b0c1bc 2023-02-13 22:01:58,896 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,897 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf1/efa49219a1404a4f8f21adbde804fa14.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,897 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/27319b9865844539846840838d43a0f9 2023-02-13 22:01:58,898 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,898 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/d22214c623124e9faa5e17aa1f6709a7 2023-02-13 22:01:58,899 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,900 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/cf2/e67902f7e0fb432c8377a88854bde947 2023-02-13 22:01:58,900 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/10a3bfd393ee45b7b9a65d3bb7040b30.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,900 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,901 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/1f18f1efc06e4748873c0469ce008ef8.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,901 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,902 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/21da36534a1140dea813aacdae9c8520.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,902 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/5108c25fa5b38236fddb2d76fe4159a5/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,903 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/recovered.edits/85.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369/recovered.edits/85.seqid 2023-02-13 22:01:58,903 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,904 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/240f909ea4d54bda9771214216b0c1bc.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,905 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/e647628404374b638b7e2af1784e5911 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf1/e647628404374b638b7e2af1784e5911 2023-02-13 22:01:58,905 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/27319b9865844539846840838d43a0f9.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,906 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/d22214c623124e9faa5e17aa1f6709a7.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,907 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/73dfc79b9fb84aa7acd9bce3ce39c6ca to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/cf2/73dfc79b9fb84aa7acd9bce3ce39c6ca 2023-02-13 22:01:58,907 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/ef0fd3f6beca364dd7ba6dfbc83ab92c/cf2/e67902f7e0fb432c8377a88854bde947.fe45e04d204e94f79d1eca4598523369 2023-02-13 22:01:58,912 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/recovered.edits/90.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5/recovered.edits/90.seqid 2023-02-13 22:01:58,912 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/recovered.edits/90.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c/recovered.edits/90.seqid 2023-02-13 22:01:58,912 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/5108c25fa5b38236fddb2d76fe4159a5 2023-02-13 22:01:58,912 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testRecoveryAndDoubleExecution/ef0fd3f6beca364dd7ba6dfbc83ab92c 2023-02-13 22:01:58,912 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 22:01:58,914 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:58,916 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-13 22:01:58,918 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-13 22:01:58,919 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:58,919 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-13 22:01:58,919 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325718919"}]},"ts":"9223372036854775807"} 2023-02-13 22:01:58,919 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325718919"}]},"ts":"9223372036854775807"} 2023-02-13 22:01:58,919 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325718919"}]},"ts":"9223372036854775807"} 2023-02-13 22:01:58,922 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 22:01:58,922 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => fe45e04d204e94f79d1eca4598523369, NAME => 'testRecoveryAndDoubleExecution,,1676325711726.fe45e04d204e94f79d1eca4598523369.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 5108c25fa5b38236fddb2d76fe4159a5, NAME => 'testRecoveryAndDoubleExecution,,1676325713678.5108c25fa5b38236fddb2d76fe4159a5.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => ef0fd3f6beca364dd7ba6dfbc83ab92c, NAME => 'testRecoveryAndDoubleExecution,41,1676325713678.ef0fd3f6beca364dd7ba6dfbc83ab92c.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 22:01:58,922 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-13 22:01:58,922 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676325718922"}]},"ts":"9223372036854775807"} 2023-02-13 22:01:58,923 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-13 22:01:58,924 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 22:01:58,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 59 msec 2023-02-13 22:01:58,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 22:01:58,970 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-13 22:01:58,991 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=376 (was 368) - Thread LEAK? -, OpenFileDescriptor=835 (was 740) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=106 (was 115), ProcessCount=162 (was 162), AvailableMemoryMB=7604 (was 7642) 2023-02-13 22:01:59,007 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=106, ProcessCount=162, AvailableMemoryMB=7603 2023-02-13 22:01:59,007 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:01:59,010 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:01:59,010 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-13 22:01:59,012 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,013 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:01:59,014 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:01:59,016 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a 2023-02-13 22:01:59,017 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a empty. 2023-02-13 22:01:59,017 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a 2023-02-13 22:01:59,017 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 22:01:59,028 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 22:01:59,030 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 16b121b90e0262812a1751ea6e14276a, NAME => 'testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:01:59,041 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:01:59,041 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 16b121b90e0262812a1751ea6e14276a, disabling compactions & flushes 2023-02-13 22:01:59,041 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,041 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,041 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. after waiting 0 ms 2023-02-13 22:01:59,041 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,041 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,041 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 16b121b90e0262812a1751ea6e14276a: Waiting for close lock at 1676325719041Disabling compacts and flushes for region at 1676325719041Disabling writes for close at 1676325719041Writing region close event to WAL at 1676325719041Closed at 1676325719041 2023-02-13 22:01:59,044 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:01:59,045 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325719045"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325719045"}]},"ts":"1676325719045"} 2023-02-13 22:01:59,047 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:01:59,048 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:01:59,048 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325719048"}]},"ts":"1676325719048"} 2023-02-13 22:01:59,050 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-13 22:01:59,053 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:01:59,054 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:01:59,054 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:01:59,054 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:01:59,054 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:01:59,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=16b121b90e0262812a1751ea6e14276a, ASSIGN}] 2023-02-13 22:01:59,056 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=16b121b90e0262812a1751ea6e14276a, ASSIGN 2023-02-13 22:01:59,057 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=16b121b90e0262812a1751ea6e14276a, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:01:59,207 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:01:59,209 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=16b121b90e0262812a1751ea6e14276a, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,210 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325719209"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325719209"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325719209"}]},"ts":"1676325719209"} 2023-02-13 22:01:59,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 16b121b90e0262812a1751ea6e14276a, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:01:59,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 16b121b90e0262812a1751ea6e14276a, NAME => 'testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:01:59,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:01:59,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:01:59,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:01:59,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:01:59,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:01:59,376 INFO [StoreOpener-16b121b90e0262812a1751ea6e14276a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:01:59,377 DEBUG [StoreOpener-16b121b90e0262812a1751ea6e14276a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1 2023-02-13 22:01:59,377 DEBUG [StoreOpener-16b121b90e0262812a1751ea6e14276a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1 2023-02-13 22:01:59,378 INFO [StoreOpener-16b121b90e0262812a1751ea6e14276a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 16b121b90e0262812a1751ea6e14276a columnFamilyName cf1 2023-02-13 22:01:59,378 INFO [StoreOpener-16b121b90e0262812a1751ea6e14276a-1] regionserver.HStore(310): Store=16b121b90e0262812a1751ea6e14276a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:59,379 INFO [StoreOpener-16b121b90e0262812a1751ea6e14276a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:01:59,380 DEBUG [StoreOpener-16b121b90e0262812a1751ea6e14276a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2 2023-02-13 22:01:59,380 DEBUG [StoreOpener-16b121b90e0262812a1751ea6e14276a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2 2023-02-13 22:01:59,380 INFO [StoreOpener-16b121b90e0262812a1751ea6e14276a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 16b121b90e0262812a1751ea6e14276a columnFamilyName cf2 2023-02-13 22:01:59,381 INFO [StoreOpener-16b121b90e0262812a1751ea6e14276a-1] regionserver.HStore(310): Store=16b121b90e0262812a1751ea6e14276a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:01:59,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a 2023-02-13 22:01:59,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a 2023-02-13 22:01:59,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:01:59,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:01:59,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:01:59,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 16b121b90e0262812a1751ea6e14276a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58986974, jitterRate=-0.12102559208869934}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:01:59,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 16b121b90e0262812a1751ea6e14276a: Running coprocessor pre-open hook at 1676325719374Writing region info on filesystem at 1676325719374Initializing all the Stores at 1676325719375 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325719375Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325719375Cleaning up temporary data from old regions at 1676325719384 (+9 ms)Cleaning up detritus from prior splits at 1676325719384Running coprocessor post-open hooks at 1676325719390 (+6 ms)Region opened successfully at 1676325719390 2023-02-13 22:01:59,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a., pid=127, masterSystemTime=1676325719365 2023-02-13 22:01:59,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,393 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=16b121b90e0262812a1751ea6e14276a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,393 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325719393"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325719393"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325719393"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325719393"}]},"ts":"1676325719393"} 2023-02-13 22:01:59,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-13 22:01:59,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 16b121b90e0262812a1751ea6e14276a, server=jenkins-hbase20.apache.org,33193,1676325615429 in 183 msec 2023-02-13 22:01:59,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-13 22:01:59,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=16b121b90e0262812a1751ea6e14276a, ASSIGN in 343 msec 2023-02-13 22:01:59,400 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:01:59,400 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325719400"}]},"ts":"1676325719400"} 2023-02-13 22:01:59,401 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-13 22:01:59,403 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:01:59,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 392 msec 2023-02-13 22:01:59,519 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:59,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,520 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:01:59,520 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,520 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 22:01:59,520 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 22:01:59,521 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,521 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,522 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,522 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,522 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,522 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,522 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,522 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:59,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,522 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,522 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:01:59,523 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:01:59,524 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:59,524 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:59,524 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,524 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. started... 2023-02-13 22:01:59,524 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 16b121b90e0262812a1751ea6e14276a 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 22:01:59,525 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,525 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,525 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,525 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:59,525 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,525 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:59,533 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/b0440b57c4d649b8b20c2e5e6871ec32 2023-02-13 22:01:59,545 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/37f19f63f3c24f13a10d11c7898597fd 2023-02-13 22:01:59,549 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/b0440b57c4d649b8b20c2e5e6871ec32 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/b0440b57c4d649b8b20c2e5e6871ec32 2023-02-13 22:01:59,555 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/b0440b57c4d649b8b20c2e5e6871ec32, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:01:59,556 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/37f19f63f3c24f13a10d11c7898597fd as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/37f19f63f3c24f13a10d11c7898597fd 2023-02-13 22:01:59,561 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/37f19f63f3c24f13a10d11c7898597fd, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:01:59,561 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 16b121b90e0262812a1751ea6e14276a in 37ms, sequenceid=6, compaction requested=false 2023-02-13 22:01:59,562 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:01:59,562 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 16b121b90e0262812a1751ea6e14276a: 2023-02-13 22:01:59,562 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,562 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:59,562 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:59,562 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,563 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:01:59,563 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:01:59,564 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,564 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:59,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:59,565 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,565 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:59,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:59,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:59,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:59,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 22:01:59,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71ef96e6[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:59,568 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 22:01:59,568 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,568 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,568 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,568 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,569 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:01:59,569 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,569 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:59,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,569 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:01:59,569 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:01:59,569 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:01:59,569 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:01:59,569 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:59,569 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:59,569 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:01:59,569 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:59,570 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:01:59,570 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,570 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:01:59,570 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,571 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:01:59,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:59,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:59,570 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:01:59,571 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,571 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,571 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:01:59,571 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:01:59,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:59,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:59,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:01:59,575 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:01:59,575 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 22:01:59,575 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:59,575 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:59,575 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,575 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:59,578 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,578 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,578 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,578 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,578 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,578 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:59,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,578 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,579 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:59,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,580 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:59,580 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:59,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,583 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,583 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:59,584 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,588 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,588 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,588 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,588 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,588 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,588 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,588 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,588 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,588 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,588 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:01:59,589 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:59,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,589 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,589 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:59,589 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,589 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,589 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:59,589 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,590 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,590 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,590 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,590 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,590 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,690 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:59,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:59,708 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:59,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:59,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,709 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:01:59,709 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,709 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 22:01:59,709 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 22:01:59,710 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,710 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,711 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,711 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,711 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,711 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,711 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,711 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:59,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:01:59,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:01:59,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,713 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:59,713 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:01:59,713 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,713 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. started... 2023-02-13 22:01:59,713 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 16b121b90e0262812a1751ea6e14276a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:59,714 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,714 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,714 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,714 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,714 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:59,714 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:59,722 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/0b37566d48664151877bf7c8a21c127d 2023-02-13 22:01:59,736 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/b8be58e6ee9244ee94525f9d6d39d795 2023-02-13 22:01:59,743 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/0b37566d48664151877bf7c8a21c127d as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/0b37566d48664151877bf7c8a21c127d 2023-02-13 22:01:59,749 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/0b37566d48664151877bf7c8a21c127d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:01:59,750 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/b8be58e6ee9244ee94525f9d6d39d795 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/b8be58e6ee9244ee94525f9d6d39d795 2023-02-13 22:01:59,756 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/b8be58e6ee9244ee94525f9d6d39d795, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:01:59,756 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 16b121b90e0262812a1751ea6e14276a in 43ms, sequenceid=19, compaction requested=false 2023-02-13 22:01:59,757 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 16b121b90e0262812a1751ea6e14276a: 2023-02-13 22:01:59,757 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,757 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:59,757 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:59,757 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,757 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:01:59,757 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:01:59,758 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,758 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:59,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:59,758 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,758 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:59,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:59,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:59,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:59,760 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 22:01:59,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52aff422[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:59,761 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 22:01:59,761 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,761 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,761 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,761 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,762 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:01:59,762 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:01:59,762 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:01:59,761 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,762 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:59,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,762 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:59,762 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:01:59,762 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:59,762 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:01:59,762 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:01:59,762 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:59,763 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:01:59,763 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,763 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:59,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:59,763 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:01:59,763 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:01:59,763 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:01:59,763 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,763 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,763 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:01:59,763 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:01:59,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:59,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:59,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:59,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:01:59,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:01:59,766 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 22:01:59,766 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:59,766 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:59,766 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,766 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:59,768 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,768 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:59,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:59,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,768 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:59,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,769 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,769 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:59,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,771 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:59,771 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,772 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,776 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,776 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,776 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,776 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,776 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,776 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,776 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,776 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,776 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,776 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:01:59,776 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:59,776 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,776 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,777 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:59,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,777 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:59,777 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,777 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,777 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,777 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,777 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,777 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,777 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,877 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:01:59,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:01:59,897 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:01:59,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:01:59,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,898 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:01:59,899 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,899 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 22:01:59,899 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 22:01:59,899 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,899 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,900 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,900 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,900 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,900 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,900 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,900 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:01:59,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,901 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,901 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:01:59,901 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:01:59,901 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,901 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,901 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:01:59,901 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:01:59,901 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,901 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:01:59,901 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:01:59,901 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:01:59,902 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:01:59,902 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:01:59,901 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,902 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. started... 2023-02-13 22:01:59,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:01:59,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,902 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,902 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 16b121b90e0262812a1751ea6e14276a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:01:59,902 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:01:59,902 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,902 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:01:59,902 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:01:59,902 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:01:59,902 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:01:59,902 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:01:59,903 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,904 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,904 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,904 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:59,904 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,904 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:59,913 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/f71e2a49c26141909e89a6ff27d9aac3 2023-02-13 22:01:59,923 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/645a6d3df7ef4a40a44faa1adc94b7ef 2023-02-13 22:01:59,926 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/f71e2a49c26141909e89a6ff27d9aac3 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f71e2a49c26141909e89a6ff27d9aac3 2023-02-13 22:01:59,929 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f71e2a49c26141909e89a6ff27d9aac3, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:01:59,930 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/645a6d3df7ef4a40a44faa1adc94b7ef as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/645a6d3df7ef4a40a44faa1adc94b7ef 2023-02-13 22:01:59,932 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/645a6d3df7ef4a40a44faa1adc94b7ef, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:01:59,933 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 16b121b90e0262812a1751ea6e14276a in 31ms, sequenceid=32, compaction requested=true 2023-02-13 22:01:59,933 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 16b121b90e0262812a1751ea6e14276a: 2023-02-13 22:01:59,933 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:01:59,933 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:01:59,933 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:01:59,933 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,933 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:01:59,933 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:01:59,934 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,934 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:59,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:59,935 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,935 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:01:59,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:59,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:59,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:59,936 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 22:01:59,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c76005b[Count = 0] remaining members to acquire global barrier 2023-02-13 22:01:59,937 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 22:01:59,937 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,937 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,937 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,937 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,937 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,937 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:01:59,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,937 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:01:59,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,937 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:01:59,937 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:01:59,937 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:01:59,938 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:01:59,937 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:01:59,937 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:01:59,938 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:01:59,938 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:01:59,938 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:01:59,938 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:01:59,938 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,938 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,938 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:01:59,938 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:01:59,938 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:01:59,939 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,939 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,939 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:01:59,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:59,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:59,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:59,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:59,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:59,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:01:59,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:01:59,941 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 22:01:59,941 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:01:59,941 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:01:59,941 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,941 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:01:59,942 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:01:59,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:01:59,942 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,942 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:01:59,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:01:59,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,943 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:01:59,944 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,944 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,944 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,946 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:01:59,946 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:01:59,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:01:59,946 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:01:59,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:01:59,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:01:59,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:01:59,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,946 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,947 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:01:59,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:01:59,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:01:59,947 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,947 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:01:59,947 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:01:59,947 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:01:59,947 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,047 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:02:00,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:02:00,069 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:02:00,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:02:00,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,071 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:02:00,071 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,071 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 22:02:00,071 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 22:02:00,072 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,072 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,073 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,073 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,073 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,073 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,073 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,073 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:02:00,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,074 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,074 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:00,074 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,075 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:00,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,075 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:00,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,075 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:02:00,075 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:02:00,075 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,075 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,075 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,075 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:02:00,075 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:02:00,075 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:02:00,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,075 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:02:00,076 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:02:00,076 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:02:00,076 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. started... 2023-02-13 22:02:00,077 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 16b121b90e0262812a1751ea6e14276a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:02:00,077 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,077 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,078 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,078 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:00,078 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,078 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:00,088 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/87a915d29b4e46b6b58f7a7572569fe3 2023-02-13 22:02:00,102 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/970baabb5e54405bb4482b9780ca2a7f 2023-02-13 22:02:00,108 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/87a915d29b4e46b6b58f7a7572569fe3 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/87a915d29b4e46b6b58f7a7572569fe3 2023-02-13 22:02:00,114 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/87a915d29b4e46b6b58f7a7572569fe3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:02:00,115 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/970baabb5e54405bb4482b9780ca2a7f as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/970baabb5e54405bb4482b9780ca2a7f 2023-02-13 22:02:00,120 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/970baabb5e54405bb4482b9780ca2a7f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:02:00,121 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 16b121b90e0262812a1751ea6e14276a in 44ms, sequenceid=45, compaction requested=true 2023-02-13 22:02:00,121 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 16b121b90e0262812a1751ea6e14276a: 2023-02-13 22:02:00,121 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:02:00,121 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:02:00,121 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:02:00,121 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,121 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:02:00,121 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:02:00,122 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,123 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:00,123 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,123 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:00,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:00,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:00,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 22:02:00,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@288ac4a5[Count = 0] remaining members to acquire global barrier 2023-02-13 22:02:00,126 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 22:02:00,126 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,127 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,127 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,127 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,127 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:02:00,128 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:02:00,128 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:02:00,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,128 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,128 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:02:00,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,128 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:02:00,128 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:02:00,128 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:02:00,128 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:02:00,128 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:02:00,128 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:02:00,129 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:02:00,129 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,129 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:02:00,130 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:02:00,130 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,130 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,130 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:02:00,130 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,130 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:02:00,130 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:02:00,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:00,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:00,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:00,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:00,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:00,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:02:00,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:02:00,135 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 22:02:00,136 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:02:00,136 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:02:00,136 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,136 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:02:00,145 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,145 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,145 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,145 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,146 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,145 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:00,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:00,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,146 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,146 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:00,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,148 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:00,149 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:00,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,153 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,153 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,154 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,156 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,157 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,157 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:02:00,157 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:02:00,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:02:00,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,157 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,158 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,158 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,158 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,158 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,158 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,158 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,158 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,184 ERROR [BP-109365581-148.251.75.209-1676325611364 heartbeating to localhost.localdomain/127.0.0.1:37463] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-109365581-148.251.75.209-1676325611364 (Datanode Uuid 3f4de7bc-de25-42a6-88c4-90335eaeb4a2) service to localhost.localdomain/127.0.0.1:37463 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 22:02:00,203 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 22:02:00,258 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:02:00,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:02:00,273 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:02:00,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:02:00,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,275 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:02:00,275 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,275 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 22:02:00,275 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 22:02:00,276 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,276 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,277 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,277 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,277 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,277 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,277 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,278 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:02:00,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:00,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:00,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:00,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,279 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:02:00,279 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:02:00,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:02:00,279 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:02:00,279 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,279 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:02:00,279 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:02:00,279 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,280 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:02:00,279 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:02:00,280 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:02:00,280 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:02:00,280 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,280 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:02:00,280 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:02:00,280 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:02:00,280 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:02:00,280 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:02:00,280 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:02:00,280 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. started... 2023-02-13 22:02:00,280 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 16b121b90e0262812a1751ea6e14276a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:02:00,281 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,281 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,281 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,281 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:00,282 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,282 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:00,290 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/f3ad22286e1442c28cde6b78d0ace30c 2023-02-13 22:02:00,305 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/c4d10acfcc2b4cd5b06b0b85fd1c5659 2023-02-13 22:02:00,308 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/f3ad22286e1442c28cde6b78d0ace30c as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f3ad22286e1442c28cde6b78d0ace30c 2023-02-13 22:02:00,311 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f3ad22286e1442c28cde6b78d0ace30c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:02:00,312 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/c4d10acfcc2b4cd5b06b0b85fd1c5659 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/c4d10acfcc2b4cd5b06b0b85fd1c5659 2023-02-13 22:02:00,315 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/c4d10acfcc2b4cd5b06b0b85fd1c5659, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:02:00,316 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 16b121b90e0262812a1751ea6e14276a in 36ms, sequenceid=58, compaction requested=true 2023-02-13 22:02:00,316 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 16b121b90e0262812a1751ea6e14276a: 2023-02-13 22:02:00,316 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:02:00,316 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:02:00,316 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:02:00,316 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,316 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:02:00,316 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:02:00,317 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,317 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:00,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:00,318 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:00,318 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:00,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:00,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:00,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 22:02:00,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25df2ff4[Count = 0] remaining members to acquire global barrier 2023-02-13 22:02:00,320 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 22:02:00,320 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,321 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,321 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,321 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,321 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,321 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:02:00,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,321 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:02:00,321 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:02:00,321 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:02:00,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,321 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:02:00,321 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:02:00,321 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:02:00,321 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:02:00,321 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:02:00,322 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:02:00,322 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,322 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:02:00,322 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:00,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:00,322 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:02:00,322 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,322 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:02:00,322 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:02:00,322 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:02:00,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:00,323 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,323 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:02:00,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:00,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:00,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:02:00,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:02:00,325 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 22:02:00,325 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:02:00,325 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:02:00,325 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,325 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:02:00,326 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,326 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,327 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,327 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,327 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,327 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,327 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,327 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,327 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,328 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:00,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:00,329 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,329 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,329 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,331 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:02:00,331 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:02:00,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,331 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:02:00,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:02:00,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,331 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,332 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,332 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,332 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,332 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,332 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,432 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:02:00,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:02:00,451 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:02:00,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 22:02:00,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,453 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:02:00,453 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,453 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 22:02:00,453 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 22:02:00,453 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,453 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,454 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,454 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,454 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,454 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,454 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,455 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:02:00,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:02:00,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:02:00,455 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 22:02:00,455 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:02:00,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,455 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:02:00,456 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,456 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 22:02:00,455 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:02:00,456 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:00,456 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:02:00,456 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 22:02:00,456 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:02:00,456 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:02:00,456 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:02:00,456 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:02:00,456 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:02:00,456 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:02:00,456 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. started... 2023-02-13 22:02:00,456 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 16b121b90e0262812a1751ea6e14276a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:02:00,457 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,457 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,458 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,458 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,458 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:00,458 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:00,465 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/5f796ee1e47b4c4b83c9d42bda5593af 2023-02-13 22:02:00,478 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/a34b3c0626c14bc9956cee40ccaa9814 2023-02-13 22:02:00,483 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/5f796ee1e47b4c4b83c9d42bda5593af as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/5f796ee1e47b4c4b83c9d42bda5593af 2023-02-13 22:02:00,489 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/5f796ee1e47b4c4b83c9d42bda5593af, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:02:00,490 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/a34b3c0626c14bc9956cee40ccaa9814 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/a34b3c0626c14bc9956cee40ccaa9814 2023-02-13 22:02:00,496 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/a34b3c0626c14bc9956cee40ccaa9814, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:02:00,497 INFO [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 16b121b90e0262812a1751ea6e14276a in 41ms, sequenceid=71, compaction requested=true 2023-02-13 22:02:00,497 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 16b121b90e0262812a1751ea6e14276a: 2023-02-13 22:02:00,497 DEBUG [rs(jenkins-hbase20.apache.org,33193,1676325615429)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:02:00,497 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:02:00,497 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:02:00,497 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,498 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 22:02:00,498 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 22:02:00,499 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,499 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:00,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:00,500 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,500 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:00,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:00,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:00,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:00,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 22:02:00,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8fcd189[Count = 0] remaining members to acquire global barrier 2023-02-13 22:02:00,503 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 22:02:00,503 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,504 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,504 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,504 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,504 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:02:00,504 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:02:00,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,504 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,504 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:02:00,504 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:02:00,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,504 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:02:00,505 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:02:00,505 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:02:00,505 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 22:02:00,505 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 22:02:00,505 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:02:00,505 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:02:00,505 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,505 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:02:00,506 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:02:00,506 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,506 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:02:00,506 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 22:02:00,506 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,506 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,506 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 22:02:00,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:00,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:00,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:00,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:00,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:00,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33193,1676325615429': 2023-02-13 22:02:00,510 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 22:02:00,510 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:02:00,510 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:02:00,510 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,510 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 22:02:00,510 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:02:00,522 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,522 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,522 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,522 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,522 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,522 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:00,523 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,523 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:00,524 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,525 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:00,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,527 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:00,528 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,528 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,532 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:00,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:00,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,532 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 22:02:00,532 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:02:00,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,533 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 22:02:00,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,533 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:02:00,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:00,532 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:00,533 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,533 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,534 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:00,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:00,534 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:00,534 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:00,534 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,534 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 22:02:00,633 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:02:00,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:02:00,651 DEBUG [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,652 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=16b121b90e0262812a1751ea6e14276a, daughterA=1b5d4d54cb017c14ba7bad13e285bd9f, daughterB=074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:00,653 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=16b121b90e0262812a1751ea6e14276a, daughterA=1b5d4d54cb017c14ba7bad13e285bd9f, daughterB=074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:00,654 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=16b121b90e0262812a1751ea6e14276a, daughterA=1b5d4d54cb017c14ba7bad13e285bd9f, daughterB=074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:00,654 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=16b121b90e0262812a1751ea6e14276a, daughterA=1b5d4d54cb017c14ba7bad13e285bd9f, daughterB=074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:00,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=16b121b90e0262812a1751ea6e14276a, UNASSIGN}] 2023-02-13 22:02:00,656 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=16b121b90e0262812a1751ea6e14276a, UNASSIGN 2023-02-13 22:02:00,657 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=16b121b90e0262812a1751ea6e14276a, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:00,657 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325720657"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325720657"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325720657"}]},"ts":"1676325720657"} 2023-02-13 22:02:00,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 16b121b90e0262812a1751ea6e14276a, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:02:00,811 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 16b121b90e0262812a1751ea6e14276a, disabling compactions & flushes 2023-02-13 22:02:00,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:02:00,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:02:00,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. after waiting 0 ms 2023-02-13 22:02:00,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:02:00,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 16b121b90e0262812a1751ea6e14276a 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 22:02:00,829 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/e8f177edf369440ab3e184e5f35eb196 2023-02-13 22:02:00,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/6cbd753fd9004106bb81187629db1dab 2023-02-13 22:02:00,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf1/e8f177edf369440ab3e184e5f35eb196 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/e8f177edf369440ab3e184e5f35eb196 2023-02-13 22:02:00,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/e8f177edf369440ab3e184e5f35eb196, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:02:00,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.tmp/cf2/6cbd753fd9004106bb81187629db1dab as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/6cbd753fd9004106bb81187629db1dab 2023-02-13 22:02:00,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/6cbd753fd9004106bb81187629db1dab, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:02:00,864 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 16b121b90e0262812a1751ea6e14276a in 52ms, sequenceid=82, compaction requested=true 2023-02-13 22:02:00,869 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 22:02:00,870 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:02:00,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. 2023-02-13 22:02:00,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 16b121b90e0262812a1751ea6e14276a: Waiting for close lock at 1676325720812Running coprocessor pre-close hooks at 1676325720812Disabling compacts and flushes for region at 1676325720812Disabling writes for close at 1676325720812Obtaining lock to block concurrent updates at 1676325720812Preparing flush snapshotting stores in 16b121b90e0262812a1751ea6e14276a at 1676325720812Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676325720813 (+1 ms)Flushing stores of testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. at 1676325720814 (+1 ms)Flushing 16b121b90e0262812a1751ea6e14276a/cf1: creating writer at 1676325720818 (+4 ms)Flushing 16b121b90e0262812a1751ea6e14276a/cf1: appending metadata at 1676325720823 (+5 ms)Flushing 16b121b90e0262812a1751ea6e14276a/cf1: closing flushed file at 1676325720823Flushing 16b121b90e0262812a1751ea6e14276a/cf2: creating writer at 1676325720834 (+11 ms)Flushing 16b121b90e0262812a1751ea6e14276a/cf2: appending metadata at 1676325720836 (+2 ms)Flushing 16b121b90e0262812a1751ea6e14276a/cf2: closing flushed file at 1676325720836Flushing 16b121b90e0262812a1751ea6e14276a/cf1: reopening flushed file at 1676325720852 (+16 ms)Flushing 16b121b90e0262812a1751ea6e14276a/cf2: reopening flushed file at 1676325720859 (+7 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 16b121b90e0262812a1751ea6e14276a in 52ms, sequenceid=82, compaction requested=true at 1676325720864 (+5 ms)Writing region close event to WAL at 1676325720866 (+2 ms)Running coprocessor post-close hooks at 1676325720870 (+4 ms)Closed at 1676325720871 (+1 ms) 2023-02-13 22:02:00,872 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,873 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=16b121b90e0262812a1751ea6e14276a, regionState=CLOSED 2023-02-13 22:02:00,873 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325720873"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325720873"}]},"ts":"1676325720873"} 2023-02-13 22:02:00,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-13 22:02:00,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 16b121b90e0262812a1751ea6e14276a, server=jenkins-hbase20.apache.org,33193,1676325615429 in 216 msec 2023-02-13 22:02:00,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-13 22:02:00,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=16b121b90e0262812a1751ea6e14276a, UNASSIGN in 220 msec 2023-02-13 22:02:00,891 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=16b121b90e0262812a1751ea6e14276a, threads=14 2023-02-13 22:02:00,891 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/37f19f63f3c24f13a10d11c7898597fd for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,891 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/645a6d3df7ef4a40a44faa1adc94b7ef for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,892 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/6cbd753fd9004106bb81187629db1dab for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,892 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/970baabb5e54405bb4482b9780ca2a7f for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,892 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/a34b3c0626c14bc9956cee40ccaa9814 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,892 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/b8be58e6ee9244ee94525f9d6d39d795 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,892 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/c4d10acfcc2b4cd5b06b0b85fd1c5659 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,892 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/0b37566d48664151877bf7c8a21c127d for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,892 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/5f796ee1e47b4c4b83c9d42bda5593af for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,892 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/87a915d29b4e46b6b58f7a7572569fe3 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,893 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/b0440b57c4d649b8b20c2e5e6871ec32 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,893 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/e8f177edf369440ab3e184e5f35eb196 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,893 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f3ad22286e1442c28cde6b78d0ace30c for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,893 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f71e2a49c26141909e89a6ff27d9aac3 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,901 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f3ad22286e1442c28cde6b78d0ace30c, top=false 2023-02-13 22:02:00,901 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/37f19f63f3c24f13a10d11c7898597fd, top=false 2023-02-13 22:02:00,901 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/c4d10acfcc2b4cd5b06b0b85fd1c5659, top=false 2023-02-13 22:02:00,902 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/645a6d3df7ef4a40a44faa1adc94b7ef, top=false 2023-02-13 22:02:00,902 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/970baabb5e54405bb4482b9780ca2a7f, top=false 2023-02-13 22:02:00,903 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/b8be58e6ee9244ee94525f9d6d39d795, top=false 2023-02-13 22:02:00,903 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/6cbd753fd9004106bb81187629db1dab, top=false 2023-02-13 22:02:00,909 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/b0440b57c4d649b8b20c2e5e6871ec32, top=false 2023-02-13 22:02:00,909 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f71e2a49c26141909e89a6ff27d9aac3, top=false 2023-02-13 22:02:00,910 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/e8f177edf369440ab3e184e5f35eb196, top=false 2023-02-13 22:02:00,910 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/0b37566d48664151877bf7c8a21c127d, top=false 2023-02-13 22:02:00,915 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/5f796ee1e47b4c4b83c9d42bda5593af, top=false 2023-02-13 22:02:00,915 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/87a915d29b4e46b6b58f7a7572569fe3, top=false 2023-02-13 22:02:00,920 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/a34b3c0626c14bc9956cee40ccaa9814, top=false 2023-02-13 22:02:00,927 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-970baabb5e54405bb4482b9780ca2a7f for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,930 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/970baabb5e54405bb4482b9780ca2a7f for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,933 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f3ad22286e1442c28cde6b78d0ace30c for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,933 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-c4d10acfcc2b4cd5b06b0b85fd1c5659 for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,933 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-e8f177edf369440ab3e184e5f35eb196 for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,934 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-a34b3c0626c14bc9956cee40ccaa9814 for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,935 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b8be58e6ee9244ee94525f9d6d39d795 for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,935 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-37f19f63f3c24f13a10d11c7898597fd for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,935 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-5f796ee1e47b4c4b83c9d42bda5593af for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,935 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-0b37566d48664151877bf7c8a21c127d for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,937 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/c4d10acfcc2b4cd5b06b0b85fd1c5659 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,938 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/e8f177edf369440ab3e184e5f35eb196 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,938 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/a34b3c0626c14bc9956cee40ccaa9814 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,938 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f3ad22286e1442c28cde6b78d0ace30c for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,939 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/37f19f63f3c24f13a10d11c7898597fd for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,939 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/b8be58e6ee9244ee94525f9d6d39d795 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,939 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/5f796ee1e47b4c4b83c9d42bda5593af for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,939 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/0b37566d48664151877bf7c8a21c127d for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,942 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-87a915d29b4e46b6b58f7a7572569fe3 for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,942 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-6cbd753fd9004106bb81187629db1dab for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,943 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b0440b57c4d649b8b20c2e5e6871ec32 for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,943 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-645a6d3df7ef4a40a44faa1adc94b7ef for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,943 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/.splits/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f71e2a49c26141909e89a6ff27d9aac3 for child: 1b5d4d54cb017c14ba7bad13e285bd9f, parent: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,945 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/b0440b57c4d649b8b20c2e5e6871ec32 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,945 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/87a915d29b4e46b6b58f7a7572569fe3 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,945 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f71e2a49c26141909e89a6ff27d9aac3 for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,945 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/6cbd753fd9004106bb81187629db1dab for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,945 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/645a6d3df7ef4a40a44faa1adc94b7ef for region: 16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:00,945 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 16b121b90e0262812a1751ea6e14276a Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-13 22:02:00,971 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:02:00,973 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:02:00,974 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676325720974"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676325720974"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676325720974"}]},"ts":"1676325720974"} 2023-02-13 22:02:00,974 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676325720974"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325720974"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325720974"}]},"ts":"1676325720974"} 2023-02-13 22:02:00,974 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676325720974"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325720974"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325720974"}]},"ts":"1676325720974"} 2023-02-13 22:02:00,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1b5d4d54cb017c14ba7bad13e285bd9f, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=074844e0735f73e3b989b530c8554ed5, ASSIGN}] 2023-02-13 22:02:00,977 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=074844e0735f73e3b989b530c8554ed5, ASSIGN 2023-02-13 22:02:00,978 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=074844e0735f73e3b989b530c8554ed5, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:02:00,978 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1b5d4d54cb017c14ba7bad13e285bd9f, ASSIGN 2023-02-13 22:02:00,979 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1b5d4d54cb017c14ba7bad13e285bd9f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,33193,1676325615429; forceNewPlan=false, retain=false 2023-02-13 22:02:01,128 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 22:02:01,131 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=1b5d4d54cb017c14ba7bad13e285bd9f, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:01,132 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676325721131"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325721131"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325721131"}]},"ts":"1676325721131"} 2023-02-13 22:02:01,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure 1b5d4d54cb017c14ba7bad13e285bd9f, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:02:01,137 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=074844e0735f73e3b989b530c8554ed5, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:01,137 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676325721137"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325721137"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325721137"}]},"ts":"1676325721137"} 2023-02-13 22:02:01,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure 074844e0735f73e3b989b530c8554ed5, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:02:01,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f. 2023-02-13 22:02:01,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1b5d4d54cb017c14ba7bad13e285bd9f, NAME => 'testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.', STARTKEY => '', ENDKEY => '71'} 2023-02-13 22:02:01,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:02:01,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:02:01,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,295 INFO [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,296 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1 2023-02-13 22:02:01,296 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1 2023-02-13 22:02:01,297 INFO [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b5d4d54cb017c14ba7bad13e285bd9f columnFamilyName cf1 2023-02-13 22:02:01,302 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-0b37566d48664151877bf7c8a21c127d 2023-02-13 22:02:01,306 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-5f796ee1e47b4c4b83c9d42bda5593af 2023-02-13 22:02:01,310 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-87a915d29b4e46b6b58f7a7572569fe3 2023-02-13 22:02:01,314 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b0440b57c4d649b8b20c2e5e6871ec32 2023-02-13 22:02:01,317 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-e8f177edf369440ab3e184e5f35eb196 2023-02-13 22:02:01,319 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f3ad22286e1442c28cde6b78d0ace30c 2023-02-13 22:02:01,321 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f71e2a49c26141909e89a6ff27d9aac3 2023-02-13 22:02:01,321 INFO [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(310): Store=1b5d4d54cb017c14ba7bad13e285bd9f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:02:01,322 INFO [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,322 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2 2023-02-13 22:02:01,322 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2 2023-02-13 22:02:01,323 INFO [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b5d4d54cb017c14ba7bad13e285bd9f columnFamilyName cf2 2023-02-13 22:02:01,326 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-37f19f63f3c24f13a10d11c7898597fd 2023-02-13 22:02:01,329 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-645a6d3df7ef4a40a44faa1adc94b7ef 2023-02-13 22:02:01,331 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-6cbd753fd9004106bb81187629db1dab 2023-02-13 22:02:01,333 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-970baabb5e54405bb4482b9780ca2a7f 2023-02-13 22:02:01,335 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-a34b3c0626c14bc9956cee40ccaa9814 2023-02-13 22:02:01,337 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b8be58e6ee9244ee94525f9d6d39d795 2023-02-13 22:02:01,339 DEBUG [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-c4d10acfcc2b4cd5b06b0b85fd1c5659 2023-02-13 22:02:01,339 INFO [StoreOpener-1b5d4d54cb017c14ba7bad13e285bd9f-1] regionserver.HStore(310): Store=1b5d4d54cb017c14ba7bad13e285bd9f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:02:01,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:02:01,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 1b5d4d54cb017c14ba7bad13e285bd9f; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75059605, jitterRate=0.11847527325153351}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:02:01,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 1b5d4d54cb017c14ba7bad13e285bd9f: Running coprocessor pre-open hook at 1676325721294Writing region info on filesystem at 1676325721294Initializing all the Stores at 1676325721295 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325721295Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325721295Cleaning up temporary data from old regions at 1676325721341 (+46 ms)Cleaning up detritus from prior splits at 1676325721341Running coprocessor post-open hooks at 1676325721343 (+2 ms)Region opened successfully at 1676325721343 2023-02-13 22:02:01,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f., pid=133, masterSystemTime=1676325721286 2023-02-13 22:02:01,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:02:01,344 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:02:01,344 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:02:01,344 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 22:02:01,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:02:01,345 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 22:02:01,345 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1912): 1b5d4d54cb017c14ba7bad13e285bd9f/cf1 is initiating minor compaction (all files) 2023-02-13 22:02:01,345 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1912): 1b5d4d54cb017c14ba7bad13e285bd9f/cf2 is initiating minor compaction (all files) 2023-02-13 22:02:01,345 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1b5d4d54cb017c14ba7bad13e285bd9f/cf1 in testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f. 2023-02-13 22:02:01,345 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1b5d4d54cb017c14ba7bad13e285bd9f/cf2 in testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f. 2023-02-13 22:02:01,345 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-37f19f63f3c24f13a10d11c7898597fd, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b8be58e6ee9244ee94525f9d6d39d795, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-645a6d3df7ef4a40a44faa1adc94b7ef, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-970baabb5e54405bb4482b9780ca2a7f, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-c4d10acfcc2b4cd5b06b0b85fd1c5659, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-a34b3c0626c14bc9956cee40ccaa9814, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-6cbd753fd9004106bb81187629db1dab] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/.tmp, totalSize=34.9 K 2023-02-13 22:02:01,345 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b0440b57c4d649b8b20c2e5e6871ec32, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-0b37566d48664151877bf7c8a21c127d, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f71e2a49c26141909e89a6ff27d9aac3, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-87a915d29b4e46b6b58f7a7572569fe3, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f3ad22286e1442c28cde6b78d0ace30c, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-5f796ee1e47b4c4b83c9d42bda5593af, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-e8f177edf369440ab3e184e5f35eb196] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/.tmp, totalSize=34.9 K 2023-02-13 22:02:01,345 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b0440b57c4d649b8b20c2e5e6871ec32, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325719515 2023-02-13 22:02:01,345 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-37f19f63f3c24f13a10d11c7898597fd, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325719515 2023-02-13 22:02:01,346 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-0b37566d48664151877bf7c8a21c127d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325719693 2023-02-13 22:02:01,346 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b8be58e6ee9244ee94525f9d6d39d795, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325719693 2023-02-13 22:02:01,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f. 2023-02-13 22:02:01,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f. 2023-02-13 22:02:01,346 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f71e2a49c26141909e89a6ff27d9aac3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325719879 2023-02-13 22:02:01,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5. 2023-02-13 22:02:01,346 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-645a6d3df7ef4a40a44faa1adc94b7ef, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325719879 2023-02-13 22:02:01,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 074844e0735f73e3b989b530c8554ed5, NAME => 'testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5.', STARTKEY => '71', ENDKEY => ''} 2023-02-13 22:02:01,346 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-87a915d29b4e46b6b58f7a7572569fe3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325720050 2023-02-13 22:02:01,346 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-970baabb5e54405bb4482b9780ca2a7f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325720050 2023-02-13 22:02:01,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:02:01,346 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=1b5d4d54cb017c14ba7bad13e285bd9f, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:01,346 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f3ad22286e1442c28cde6b78d0ace30c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325720259 2023-02-13 22:02:01,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,347 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-c4d10acfcc2b4cd5b06b0b85fd1c5659, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325720259 2023-02-13 22:02:01,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:02:01,347 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676325721346"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325721346"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325721346"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325721346"}]},"ts":"1676325721346"} 2023-02-13 22:02:01,347 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-5f796ee1e47b4c4b83c9d42bda5593af, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325720435 2023-02-13 22:02:01,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,347 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-a34b3c0626c14bc9956cee40ccaa9814, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325720435 2023-02-13 22:02:01,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,347 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-e8f177edf369440ab3e184e5f35eb196, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325720637 2023-02-13 22:02:01,347 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-6cbd753fd9004106bb81187629db1dab, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325720637 2023-02-13 22:02:01,348 INFO [StoreOpener-074844e0735f73e3b989b530c8554ed5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,350 DEBUG [StoreOpener-074844e0735f73e3b989b530c8554ed5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/cf1 2023-02-13 22:02:01,350 DEBUG [StoreOpener-074844e0735f73e3b989b530c8554ed5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/cf1 2023-02-13 22:02:01,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-02-13 22:02:01,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure 1b5d4d54cb017c14ba7bad13e285bd9f, server=jenkins-hbase20.apache.org,33193,1676325615429 in 215 msec 2023-02-13 22:02:01,351 INFO [StoreOpener-074844e0735f73e3b989b530c8554ed5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 074844e0735f73e3b989b530c8554ed5 columnFamilyName cf1 2023-02-13 22:02:01,351 INFO [StoreOpener-074844e0735f73e3b989b530c8554ed5-1] regionserver.HStore(310): Store=074844e0735f73e3b989b530c8554ed5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:02:01,351 INFO [StoreOpener-074844e0735f73e3b989b530c8554ed5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1b5d4d54cb017c14ba7bad13e285bd9f, ASSIGN in 373 msec 2023-02-13 22:02:01,353 DEBUG [StoreOpener-074844e0735f73e3b989b530c8554ed5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/cf2 2023-02-13 22:02:01,353 DEBUG [StoreOpener-074844e0735f73e3b989b530c8554ed5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/cf2 2023-02-13 22:02:01,353 INFO [StoreOpener-074844e0735f73e3b989b530c8554ed5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 074844e0735f73e3b989b530c8554ed5 columnFamilyName cf2 2023-02-13 22:02:01,354 INFO [StoreOpener-074844e0735f73e3b989b530c8554ed5-1] regionserver.HStore(310): Store=074844e0735f73e3b989b530c8554ed5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:02:01,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:02:01,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 074844e0735f73e3b989b530c8554ed5; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69650470, jitterRate=0.037872880697250366}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:02:01,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 074844e0735f73e3b989b530c8554ed5: Running coprocessor pre-open hook at 1676325721347Writing region info on filesystem at 1676325721347Initializing all the Stores at 1676325721348 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325721348Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325721348Cleaning up temporary data from old regions at 1676325721356 (+8 ms)Cleaning up detritus from prior splits at 1676325721356Running coprocessor post-open hooks at 1676325721359 (+3 ms)Region opened successfully at 1676325721359 2023-02-13 22:02:01,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5., pid=134, masterSystemTime=1676325721286 2023-02-13 22:02:01,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5. 2023-02-13 22:02:01,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5. 2023-02-13 22:02:01,360 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=074844e0735f73e3b989b530c8554ed5, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:01,360 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676325721360"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325721360"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325721360"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325721360"}]},"ts":"1676325721360"} 2023-02-13 22:02:01,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-02-13 22:02:01,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure 074844e0735f73e3b989b530c8554ed5, server=jenkins-hbase20.apache.org,33193,1676325615429 in 222 msec 2023-02-13 22:02:01,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-13 22:02:01,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=074844e0735f73e3b989b530c8554ed5, ASSIGN in 385 msec 2023-02-13 22:02:01,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=16b121b90e0262812a1751ea6e14276a, daughterA=1b5d4d54cb017c14ba7bad13e285bd9f, daughterB=074844e0735f73e3b989b530c8554ed5 in 712 msec 2023-02-13 22:02:01,371 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1b5d4d54cb017c14ba7bad13e285bd9f#cf1#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:02:01,371 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] throttle.PressureAwareThroughputController(145): 1b5d4d54cb017c14ba7bad13e285bd9f#cf2#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:02:01,379 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/.tmp/cf1/3ae806815e2c43a9a9aa777bd8809fd8 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/3ae806815e2c43a9a9aa777bd8809fd8 2023-02-13 22:02:01,379 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/.tmp/cf2/1ea197dc4fde4356a0e0952686028111 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/1ea197dc4fde4356a0e0952686028111 2023-02-13 22:02:01,382 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1b5d4d54cb017c14ba7bad13e285bd9f/cf2 of 1b5d4d54cb017c14ba7bad13e285bd9f into 1ea197dc4fde4356a0e0952686028111(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:02:01,382 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1b5d4d54cb017c14ba7bad13e285bd9f/cf1 of 1b5d4d54cb017c14ba7bad13e285bd9f into 3ae806815e2c43a9a9aa777bd8809fd8(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:02:01,382 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1b5d4d54cb017c14ba7bad13e285bd9f: 2023-02-13 22:02:01,382 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1b5d4d54cb017c14ba7bad13e285bd9f: 2023-02-13 22:02:01,382 INFO [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f., storeName=1b5d4d54cb017c14ba7bad13e285bd9f/cf2, priority=9, startTime=1676325721344; duration=0sec 2023-02-13 22:02:01,382 INFO [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f., storeName=1b5d4d54cb017c14ba7bad13e285bd9f/cf1, priority=9, startTime=1676325721344; duration=0sec 2023-02-13 22:02:01,382 DEBUG [RS:1;jenkins-hbase20:33193-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:02:01,382 DEBUG [RS:1;jenkins-hbase20:33193-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:02:01,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33193] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 148.251.75.209:49504 deadline: 1676325781403, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a. is not online on jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:01,524 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-13 22:02:01,530 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:02:01,533 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-13 22:02:01,533 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitTableRegionEmptyDaughter 2023-02-13 22:02:01,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 22:02:01,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 22:02:01,538 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325721538"}]},"ts":"1676325721538"} 2023-02-13 22:02:01,539 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-13 22:02:01,540 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-13 22:02:01,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1b5d4d54cb017c14ba7bad13e285bd9f, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=074844e0735f73e3b989b530c8554ed5, UNASSIGN}] 2023-02-13 22:02:01,541 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=074844e0735f73e3b989b530c8554ed5, UNASSIGN 2023-02-13 22:02:01,542 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=074844e0735f73e3b989b530c8554ed5, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:01,542 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676325721542"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325721542"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325721542"}]},"ts":"1676325721542"} 2023-02-13 22:02:01,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 074844e0735f73e3b989b530c8554ed5, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:02:01,544 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1b5d4d54cb017c14ba7bad13e285bd9f, UNASSIGN 2023-02-13 22:02:01,545 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=1b5d4d54cb017c14ba7bad13e285bd9f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:01,545 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676325721545"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325721545"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325721545"}]},"ts":"1676325721545"} 2023-02-13 22:02:01,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 1b5d4d54cb017c14ba7bad13e285bd9f, server=jenkins-hbase20.apache.org,33193,1676325615429}] 2023-02-13 22:02:01,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 22:02:01,694 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 074844e0735f73e3b989b530c8554ed5, disabling compactions & flushes 2023-02-13 22:02:01,695 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5. 2023-02-13 22:02:01,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5. 2023-02-13 22:02:01,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5. after waiting 0 ms 2023-02-13 22:02:01,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5. 2023-02-13 22:02:01,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-13 22:02:01,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:02:01,700 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5. 2023-02-13 22:02:01,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 074844e0735f73e3b989b530c8554ed5: Waiting for close lock at 1676325721694Running coprocessor pre-close hooks at 1676325721694Disabling compacts and flushes for region at 1676325721694Disabling writes for close at 1676325721695 (+1 ms)Writing region close event to WAL at 1676325721696 (+1 ms)Running coprocessor post-close hooks at 1676325721700 (+4 ms)Closed at 1676325721700 2023-02-13 22:02:01,701 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,701 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,701 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 1b5d4d54cb017c14ba7bad13e285bd9f, disabling compactions & flushes 2023-02-13 22:02:01,701 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f. 2023-02-13 22:02:01,701 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f. 2023-02-13 22:02:01,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f. after waiting 0 ms 2023-02-13 22:02:01,702 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=074844e0735f73e3b989b530c8554ed5, regionState=CLOSED 2023-02-13 22:02:01,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f. 2023-02-13 22:02:01,702 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676325721702"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325721702"}]},"ts":"1676325721702"} 2023-02-13 22:02:01,702 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b0440b57c4d649b8b20c2e5e6871ec32, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-0b37566d48664151877bf7c8a21c127d, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f71e2a49c26141909e89a6ff27d9aac3, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-87a915d29b4e46b6b58f7a7572569fe3, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f3ad22286e1442c28cde6b78d0ace30c, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-5f796ee1e47b4c4b83c9d42bda5593af, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-e8f177edf369440ab3e184e5f35eb196] to archive 2023-02-13 22:02:01,703 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:02:01,705 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b0440b57c4d649b8b20c2e5e6871ec32 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b0440b57c4d649b8b20c2e5e6871ec32 2023-02-13 22:02:01,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-13 22:02:01,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 074844e0735f73e3b989b530c8554ed5, server=jenkins-hbase20.apache.org,33193,1676325615429 in 161 msec 2023-02-13 22:02:01,707 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-0b37566d48664151877bf7c8a21c127d to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-0b37566d48664151877bf7c8a21c127d 2023-02-13 22:02:01,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=074844e0735f73e3b989b530c8554ed5, UNASSIGN in 166 msec 2023-02-13 22:02:01,708 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f71e2a49c26141909e89a6ff27d9aac3 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f71e2a49c26141909e89a6ff27d9aac3 2023-02-13 22:02:01,709 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-87a915d29b4e46b6b58f7a7572569fe3 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-87a915d29b4e46b6b58f7a7572569fe3 2023-02-13 22:02:01,711 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f3ad22286e1442c28cde6b78d0ace30c to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-f3ad22286e1442c28cde6b78d0ace30c 2023-02-13 22:02:01,712 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-5f796ee1e47b4c4b83c9d42bda5593af to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-5f796ee1e47b4c4b83c9d42bda5593af 2023-02-13 22:02:01,713 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-e8f177edf369440ab3e184e5f35eb196 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-e8f177edf369440ab3e184e5f35eb196 2023-02-13 22:02:01,715 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-37f19f63f3c24f13a10d11c7898597fd, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b8be58e6ee9244ee94525f9d6d39d795, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-645a6d3df7ef4a40a44faa1adc94b7ef, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-970baabb5e54405bb4482b9780ca2a7f, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-c4d10acfcc2b4cd5b06b0b85fd1c5659, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-a34b3c0626c14bc9956cee40ccaa9814, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-6cbd753fd9004106bb81187629db1dab] to archive 2023-02-13 22:02:01,716 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:02:01,717 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-37f19f63f3c24f13a10d11c7898597fd to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-37f19f63f3c24f13a10d11c7898597fd 2023-02-13 22:02:01,718 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b8be58e6ee9244ee94525f9d6d39d795 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-b8be58e6ee9244ee94525f9d6d39d795 2023-02-13 22:02:01,719 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-645a6d3df7ef4a40a44faa1adc94b7ef to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-645a6d3df7ef4a40a44faa1adc94b7ef 2023-02-13 22:02:01,720 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-970baabb5e54405bb4482b9780ca2a7f to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-970baabb5e54405bb4482b9780ca2a7f 2023-02-13 22:02:01,720 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-c4d10acfcc2b4cd5b06b0b85fd1c5659 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-c4d10acfcc2b4cd5b06b0b85fd1c5659 2023-02-13 22:02:01,721 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-a34b3c0626c14bc9956cee40ccaa9814 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-a34b3c0626c14bc9956cee40ccaa9814 2023-02-13 22:02:01,722 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-6cbd753fd9004106bb81187629db1dab to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/testSplitTableRegionEmptyDaughter=16b121b90e0262812a1751ea6e14276a-6cbd753fd9004106bb81187629db1dab 2023-02-13 22:02:01,724 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 22:02:01,724 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:02:01,724 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f. 2023-02-13 22:02:01,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 1b5d4d54cb017c14ba7bad13e285bd9f: Waiting for close lock at 1676325721701Running coprocessor pre-close hooks at 1676325721701Disabling compacts and flushes for region at 1676325721701Disabling writes for close at 1676325721702 (+1 ms)Writing region close event to WAL at 1676325721722 (+20 ms)Running coprocessor post-close hooks at 1676325721724 (+2 ms)Closed at 1676325721724 2023-02-13 22:02:01,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,726 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=1b5d4d54cb017c14ba7bad13e285bd9f, regionState=CLOSED 2023-02-13 22:02:01,726 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676325721726"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325721726"}]},"ts":"1676325721726"} 2023-02-13 22:02:01,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-13 22:02:01,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 1b5d4d54cb017c14ba7bad13e285bd9f, server=jenkins-hbase20.apache.org,33193,1676325615429 in 181 msec 2023-02-13 22:02:01,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-02-13 22:02:01,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1b5d4d54cb017c14ba7bad13e285bd9f, UNASSIGN in 187 msec 2023-02-13 22:02:01,731 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325721731"}]},"ts":"1676325721731"} 2023-02-13 22:02:01,732 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-13 22:02:01,733 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-13 22:02:01,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 201 msec 2023-02-13 22:02:01,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 22:02:01,841 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-13 22:02:01,842 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitTableRegionEmptyDaughter 2023-02-13 22:02:01,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 22:02:01,845 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 22:02:01,847 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 22:02:01,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 22:02:01,854 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:01,854 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,854 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,858 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/recovered.edits] 2023-02-13 22:02:01,858 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/recovered.edits] 2023-02-13 22:02:01,858 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/recovered.edits] 2023-02-13 22:02:01,860 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/0b37566d48664151877bf7c8a21c127d to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/0b37566d48664151877bf7c8a21c127d 2023-02-13 22:02:01,860 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/3ae806815e2c43a9a9aa777bd8809fd8 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf1/3ae806815e2c43a9a9aa777bd8809fd8 2023-02-13 22:02:01,861 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/5f796ee1e47b4c4b83c9d42bda5593af to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/5f796ee1e47b4c4b83c9d42bda5593af 2023-02-13 22:02:01,862 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/recovered.edits/88.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5/recovered.edits/88.seqid 2023-02-13 22:02:01,862 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/1ea197dc4fde4356a0e0952686028111 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/cf2/1ea197dc4fde4356a0e0952686028111 2023-02-13 22:02:01,862 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/87a915d29b4e46b6b58f7a7572569fe3 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/87a915d29b4e46b6b58f7a7572569fe3 2023-02-13 22:02:01,862 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/074844e0735f73e3b989b530c8554ed5 2023-02-13 22:02:01,863 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/b0440b57c4d649b8b20c2e5e6871ec32 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/b0440b57c4d649b8b20c2e5e6871ec32 2023-02-13 22:02:01,864 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/e8f177edf369440ab3e184e5f35eb196 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/e8f177edf369440ab3e184e5f35eb196 2023-02-13 22:02:01,864 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/recovered.edits/90.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f/recovered.edits/90.seqid 2023-02-13 22:02:01,864 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/1b5d4d54cb017c14ba7bad13e285bd9f 2023-02-13 22:02:01,865 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f3ad22286e1442c28cde6b78d0ace30c to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f3ad22286e1442c28cde6b78d0ace30c 2023-02-13 22:02:01,865 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f71e2a49c26141909e89a6ff27d9aac3 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf1/f71e2a49c26141909e89a6ff27d9aac3 2023-02-13 22:02:01,867 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/37f19f63f3c24f13a10d11c7898597fd to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/37f19f63f3c24f13a10d11c7898597fd 2023-02-13 22:02:01,868 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/645a6d3df7ef4a40a44faa1adc94b7ef to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/645a6d3df7ef4a40a44faa1adc94b7ef 2023-02-13 22:02:01,870 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/6cbd753fd9004106bb81187629db1dab to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/6cbd753fd9004106bb81187629db1dab 2023-02-13 22:02:01,871 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/970baabb5e54405bb4482b9780ca2a7f to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/970baabb5e54405bb4482b9780ca2a7f 2023-02-13 22:02:01,872 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/a34b3c0626c14bc9956cee40ccaa9814 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/a34b3c0626c14bc9956cee40ccaa9814 2023-02-13 22:02:01,873 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/b8be58e6ee9244ee94525f9d6d39d795 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/b8be58e6ee9244ee94525f9d6d39d795 2023-02-13 22:02:01,873 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/c4d10acfcc2b4cd5b06b0b85fd1c5659 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/cf2/c4d10acfcc2b4cd5b06b0b85fd1c5659 2023-02-13 22:02:01,875 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/recovered.edits/85.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a/recovered.edits/85.seqid 2023-02-13 22:02:01,876 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegionEmptyDaughter/16b121b90e0262812a1751ea6e14276a 2023-02-13 22:02:01,876 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 22:02:01,877 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 22:02:01,879 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-13 22:02:01,881 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-13 22:02:01,882 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 22:02:01,882 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-13 22:02:01,882 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325721882"}]},"ts":"9223372036854775807"} 2023-02-13 22:02:01,882 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325721882"}]},"ts":"9223372036854775807"} 2023-02-13 22:02:01,882 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325721882"}]},"ts":"9223372036854775807"} 2023-02-13 22:02:01,884 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 22:02:01,884 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 16b121b90e0262812a1751ea6e14276a, NAME => 'testSplitTableRegionEmptyDaughter,,1676325719011.16b121b90e0262812a1751ea6e14276a.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 1b5d4d54cb017c14ba7bad13e285bd9f, NAME => 'testSplitTableRegionEmptyDaughter,,1676325720651.1b5d4d54cb017c14ba7bad13e285bd9f.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 074844e0735f73e3b989b530c8554ed5, NAME => 'testSplitTableRegionEmptyDaughter,71,1676325720651.074844e0735f73e3b989b530c8554ed5.', STARTKEY => '71', ENDKEY => ''}] 2023-02-13 22:02:01,884 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-13 22:02:01,884 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676325721884"}]},"ts":"9223372036854775807"} 2023-02-13 22:02:01,885 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-13 22:02:01,886 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 22:02:01,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 44 msec 2023-02-13 22:02:01,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 22:02:01,948 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-13 22:02:01,958 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 376), OpenFileDescriptor=833 (was 835), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=98 (was 106), ProcessCount=162 (was 162), AvailableMemoryMB=7603 (was 7603) 2023-02-13 22:02:01,969 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=833, MaxFileDescriptor=60000, SystemLoadAverage=98, ProcessCount=162, AvailableMemoryMB=7602 2023-02-13 22:02:01,969 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:02:01,971 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:02:01,972 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-13 22:02:01,973 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-13 22:02:01,974 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:02:01,975 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:02:01,976 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:01,977 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e empty. 2023-02-13 22:02:01,977 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:01,977 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 22:02:01,988 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-13 22:02:01,989 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => d916f52616da598e54eb3efe0a77314e, NAME => 'testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp 2023-02-13 22:02:01,998 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:02:01,999 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing d916f52616da598e54eb3efe0a77314e, disabling compactions & flushes 2023-02-13 22:02:01,999 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:01,999 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:01,999 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. after waiting 0 ms 2023-02-13 22:02:01,999 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:01,999 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:01,999 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for d916f52616da598e54eb3efe0a77314e: Waiting for close lock at 1676325721999Disabling compacts and flushes for region at 1676325721999Disabling writes for close at 1676325721999Writing region close event to WAL at 1676325721999Closed at 1676325721999 2023-02-13 22:02:02,000 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:02:02,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325722001"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325722001"}]},"ts":"1676325722001"} 2023-02-13 22:02:02,002 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:02:02,003 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:02:02,003 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325722003"}]},"ts":"1676325722003"} 2023-02-13 22:02:02,003 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-13 22:02:02,005 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:02:02,006 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:02:02,006 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:02:02,006 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:02:02,006 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:02:02,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=d916f52616da598e54eb3efe0a77314e, ASSIGN}] 2023-02-13 22:02:02,007 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=d916f52616da598e54eb3efe0a77314e, ASSIGN 2023-02-13 22:02:02,008 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=d916f52616da598e54eb3efe0a77314e, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37979,1676325615388; forceNewPlan=false, retain=false 2023-02-13 22:02:02,158 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:02:02,160 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=d916f52616da598e54eb3efe0a77314e, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,160 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325722160"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325722160"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325722160"}]},"ts":"1676325722160"} 2023-02-13 22:02:02,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure d916f52616da598e54eb3efe0a77314e, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:02:02,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:02,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d916f52616da598e54eb3efe0a77314e, NAME => 'testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:02:02,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:02:02,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:02,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:02:02,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:02,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:02,331 INFO [StoreOpener-d916f52616da598e54eb3efe0a77314e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:02,333 DEBUG [StoreOpener-d916f52616da598e54eb3efe0a77314e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1 2023-02-13 22:02:02,333 DEBUG [StoreOpener-d916f52616da598e54eb3efe0a77314e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1 2023-02-13 22:02:02,333 INFO [StoreOpener-d916f52616da598e54eb3efe0a77314e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d916f52616da598e54eb3efe0a77314e columnFamilyName cf1 2023-02-13 22:02:02,334 INFO [StoreOpener-d916f52616da598e54eb3efe0a77314e-1] regionserver.HStore(310): Store=d916f52616da598e54eb3efe0a77314e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:02:02,334 INFO [StoreOpener-d916f52616da598e54eb3efe0a77314e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:02,335 DEBUG [StoreOpener-d916f52616da598e54eb3efe0a77314e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2 2023-02-13 22:02:02,335 DEBUG [StoreOpener-d916f52616da598e54eb3efe0a77314e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2 2023-02-13 22:02:02,335 INFO [StoreOpener-d916f52616da598e54eb3efe0a77314e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d916f52616da598e54eb3efe0a77314e columnFamilyName cf2 2023-02-13 22:02:02,336 INFO [StoreOpener-d916f52616da598e54eb3efe0a77314e-1] regionserver.HStore(310): Store=d916f52616da598e54eb3efe0a77314e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:02:02,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:02,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:02,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:02:02,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:02,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:02:02,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened d916f52616da598e54eb3efe0a77314e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65427424, jitterRate=-0.025055408477783203}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:02:02,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for d916f52616da598e54eb3efe0a77314e: Running coprocessor pre-open hook at 1676325722330Writing region info on filesystem at 1676325722330Initializing all the Stores at 1676325722331 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325722331Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325722331Cleaning up temporary data from old regions at 1676325722338 (+7 ms)Cleaning up detritus from prior splits at 1676325722338Running coprocessor post-open hooks at 1676325722343 (+5 ms)Region opened successfully at 1676325722343 2023-02-13 22:02:02,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e., pid=143, masterSystemTime=1676325722316 2023-02-13 22:02:02,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:02,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:02,346 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=d916f52616da598e54eb3efe0a77314e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,346 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325722346"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325722346"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325722346"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325722346"}]},"ts":"1676325722346"} 2023-02-13 22:02:02,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-13 22:02:02,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure d916f52616da598e54eb3efe0a77314e, server=jenkins-hbase20.apache.org,37979,1676325615388 in 184 msec 2023-02-13 22:02:02,351 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-13 22:02:02,351 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=d916f52616da598e54eb3efe0a77314e, ASSIGN in 343 msec 2023-02-13 22:02:02,351 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:02:02,351 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325722351"}]},"ts":"1676325722351"} 2023-02-13 22:02:02,353 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-13 22:02:02,355 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:02:02,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 383 msec 2023-02-13 22:02:02,487 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:02:02,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 22:02:02,489 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 22:02:02,489 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,489 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 22:02:02,489 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 22:02:02,490 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,490 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,491 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,492 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,492 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,492 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,492 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,492 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:02:02,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,494 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:02,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,494 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:02,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,495 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:02,494 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:02,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:02,495 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:02,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,496 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:02,496 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,496 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:02,496 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:02,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:02,496 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:02,496 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:02,496 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:02,496 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,496 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:02,496 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:02,496 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:02,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:02,497 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:02,497 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,497 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:02,497 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:02:02,497 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:02:02,497 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:02,497 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. started... 2023-02-13 22:02:02,499 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d916f52616da598e54eb3efe0a77314e 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 22:02:02,499 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,499 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,499 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,500 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:02,500 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,500 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:02,508 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/cbbec0e884ef47bfb33c98cac8b97b33 2023-02-13 22:02:02,525 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/c317f00b85434aa48d57983ab25c22a5 2023-02-13 22:02:02,529 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/cbbec0e884ef47bfb33c98cac8b97b33 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33 2023-02-13 22:02:02,532 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:02:02,532 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/c317f00b85434aa48d57983ab25c22a5 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5 2023-02-13 22:02:02,535 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:02:02,536 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d916f52616da598e54eb3efe0a77314e in 37ms, sequenceid=6, compaction requested=false 2023-02-13 22:02:02,536 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-13 22:02:02,536 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d916f52616da598e54eb3efe0a77314e: 2023-02-13 22:02:02,536 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:02,536 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:02:02,536 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:02:02,536 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,536 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:02,536 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:02,537 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,537 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:02,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:02,537 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,538 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:02,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:02,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:02,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:02,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 22:02:02,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24246c15[Count = 0] remaining members to acquire global barrier 2023-02-13 22:02:02,540 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 22:02:02,540 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,542 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,542 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,542 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,542 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:02,542 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,542 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:02:02,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,542 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:02,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,542 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:02:02,542 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:02,542 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:02,542 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:02,542 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:02,542 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:02:02,542 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:02:02,543 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:02,543 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,543 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:02,543 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,544 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:02,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:02,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:02,544 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:02,544 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,544 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,544 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:02,544 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:02,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:02,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:02,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:02,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:02:02,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 22:02:02,547 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 22:02:02,547 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:02:02,547 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:02:02,547 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 22:02:02,547 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:02:02,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,556 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:02,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:02,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,555 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,556 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:02,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,557 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,558 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:02,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:02,561 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,561 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,562 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,565 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,565 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,565 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,565 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,565 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,565 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 22:02:02,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:02:02,565 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,565 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,565 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,565 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,566 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 22:02:02,566 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,566 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:02:02,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,566 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,566 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,567 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,567 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,567 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,567 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,567 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,666 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:02:02,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:02:02,688 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:02:02,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 22:02:02,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 22:02:02,690 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 22:02:02,690 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,690 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 22:02:02,690 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 22:02:02,691 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,691 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,692 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,692 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,692 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,692 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,693 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,693 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:02:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,694 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,694 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,694 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,695 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:02,695 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:02,695 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:02,695 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:02,695 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:02,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,695 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,695 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:02,695 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:02,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:02,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:02,697 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:02,697 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,697 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:02,697 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:02:02,697 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:02:02,697 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:02,697 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. started... 2023-02-13 22:02:02,697 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d916f52616da598e54eb3efe0a77314e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:02:02,705 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,705 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,705 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,705 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:02,705 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,706 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:02,708 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/0c07e9e45e564156b9aeca4d60a7e068 2023-02-13 22:02:02,719 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/73af04a2269445a5aa92ce2079b92209 2023-02-13 22:02:02,722 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/0c07e9e45e564156b9aeca4d60a7e068 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068 2023-02-13 22:02:02,725 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:02:02,726 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/73af04a2269445a5aa92ce2079b92209 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209 2023-02-13 22:02:02,728 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 22:02:02,729 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d916f52616da598e54eb3efe0a77314e in 32ms, sequenceid=19, compaction requested=false 2023-02-13 22:02:02,729 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d916f52616da598e54eb3efe0a77314e: 2023-02-13 22:02:02,729 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:02,729 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:02:02,729 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:02:02,729 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,729 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:02,729 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:02,730 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,730 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:02,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:02,730 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,731 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:02,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 22:02:02,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a752bd8[Count = 0] remaining members to acquire global barrier 2023-02-13 22:02:02,732 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 22:02:02,732 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,732 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,732 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,732 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,733 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,733 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:02:02,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,733 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:02,733 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:02,733 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:02,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,733 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:02:02,733 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:02,733 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:02,733 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:02,733 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:02:02,733 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:02:02,734 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:02,734 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,734 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:02,734 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:02,734 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,734 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:02,734 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:02,734 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,734 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,734 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:02,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:02,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:02,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:02,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:02,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:02,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:02:02,737 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 22:02:02,737 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 22:02:02,737 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:02:02,737 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:02:02,737 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 22:02:02,737 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:02:02,738 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,738 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,738 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,738 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,738 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,738 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,738 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,738 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:02,739 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,739 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:02,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:02,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,740 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,741 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:02,741 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,743 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,743 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,743 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,743 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 22:02:02,743 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:02:02,743 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,743 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,743 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,743 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,743 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:02:02,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,744 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,844 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:02:02,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:02:02,867 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:02:02,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 22:02:02,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 22:02:02,869 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 22:02:02,869 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,869 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 22:02:02,869 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 22:02:02,870 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,870 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,870 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,871 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,870 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,871 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,871 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,871 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:02:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:02,872 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:02,873 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:02,873 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:02,873 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:02,873 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:02,873 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,873 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:02,873 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:02:02,874 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:02:02,874 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:02,874 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. started... 2023-02-13 22:02:02,874 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d916f52616da598e54eb3efe0a77314e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:02:02,874 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,875 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,875 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:02,875 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,875 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,875 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:02,889 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/48237147796140ddbd944793d35ae114 2023-02-13 22:02:02,899 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/3d4fd1ca001041ef84257eb61351ad10 2023-02-13 22:02:02,902 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/48237147796140ddbd944793d35ae114 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114 2023-02-13 22:02:02,905 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:02:02,906 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/3d4fd1ca001041ef84257eb61351ad10 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10 2023-02-13 22:02:02,908 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 22:02:02,909 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d916f52616da598e54eb3efe0a77314e in 35ms, sequenceid=32, compaction requested=true 2023-02-13 22:02:02,909 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d916f52616da598e54eb3efe0a77314e: 2023-02-13 22:02:02,909 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:02,909 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:02:02,909 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:02:02,909 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,909 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:02,909 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:02,910 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,910 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:02,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:02,911 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,911 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:02,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:02,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:02,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:02,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 22:02:02,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6891fed[Count = 0] remaining members to acquire global barrier 2023-02-13 22:02:02,912 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 22:02:02,912 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,913 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,913 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,913 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,913 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:02,913 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,913 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:02:02,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,913 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:02,913 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:02,913 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:02:02,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,913 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:02,913 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:02,913 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:02:02,913 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:02,913 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:02:02,914 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:02,914 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,914 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:02,914 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,914 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:02,914 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:02,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:02,914 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:02,914 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:02,914 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,914 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:02,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:02:02,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 22:02:02,917 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 22:02:02,917 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:02:02,917 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:02:02,917 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 22:02:02,917 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:02:02,917 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,917 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,917 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:02,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,917 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,917 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,917 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:02,918 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,918 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,918 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:02,918 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:02,919 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:02,920 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,920 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:02,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,921 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 22:02:02,922 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:02:02,921 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:02,921 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:02,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:02,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,922 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 22:02:02,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,922 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:02,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:02,922 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,922 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:02:02,922 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:02,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:02,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,922 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:02,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:02,922 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:02,922 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,022 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:02:03,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:02:03,046 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:02:03,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 22:02:03,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 22:02:03,048 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 22:02:03,048 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,048 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 22:02:03,048 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 22:02:03,049 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,049 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,050 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,050 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,050 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,050 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:02:03,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:03,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,051 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:03,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:03,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:03,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:03,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:03,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:03,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:03,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:02:03,052 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:02:03,052 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:03,053 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,053 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. started... 2023-02-13 22:02:03,053 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:03,053 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:03,053 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:03,053 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d916f52616da598e54eb3efe0a77314e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:02:03,053 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,054 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,054 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,054 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:03,054 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,054 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:03,063 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c 2023-02-13 22:02:03,077 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/94e4e89845964aa199e34d446dfbee8a 2023-02-13 22:02:03,080 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c 2023-02-13 22:02:03,083 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:02:03,084 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/94e4e89845964aa199e34d446dfbee8a as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a 2023-02-13 22:02:03,087 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 22:02:03,087 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d916f52616da598e54eb3efe0a77314e in 34ms, sequenceid=45, compaction requested=true 2023-02-13 22:02:03,087 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d916f52616da598e54eb3efe0a77314e: 2023-02-13 22:02:03,087 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:03,088 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:02:03,088 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:02:03,088 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,088 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:03,088 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:03,089 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,089 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:03,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:03,089 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,089 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:03,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:03,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:03,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:03,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 22:02:03,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6897e62c[Count = 0] remaining members to acquire global barrier 2023-02-13 22:02:03,090 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 22:02:03,090 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,091 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,091 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,091 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,091 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:03,091 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:03,091 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:03,091 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,091 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:02:03,091 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:03,091 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:02:03,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,091 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:02:03,091 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:03,091 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:03,091 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:02:03,092 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:03,092 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,092 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:03,093 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:03,093 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:03,093 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,093 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:03,093 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,093 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:03,093 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:02:03,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 22:02:03,096 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 22:02:03,096 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:02:03,096 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:02:03,096 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 22:02:03,096 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:02:03,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:03,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,097 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:03,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,098 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,098 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,100 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:03,100 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,100 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,102 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 22:02:03,102 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:02:03,102 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,103 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,102 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,103 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,103 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,103 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 22:02:03,103 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,103 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:02:03,103 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,183 ERROR [BP-109365581-148.251.75.209-1676325611364 heartbeating to localhost.localdomain/127.0.0.1:37463] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-109365581-148.251.75.209-1676325611364 (Datanode Uuid 643297d9-b5a4-472e-97c3-eb890073d22e) service to localhost.localdomain/127.0.0.1:37463 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 22:02:03,203 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:02:03,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:02:03,219 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:02:03,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 22:02:03,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 22:02:03,221 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 22:02:03,221 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,221 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 22:02:03,221 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 22:02:03,222 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,222 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,223 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,223 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,223 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,223 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,224 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,224 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:02:03,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:03,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:03,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:03,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:03,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:03,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,226 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:03,226 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:03,226 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:03,226 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:03,226 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:03,226 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:03,226 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:03,226 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,226 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:03,226 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:03,226 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:03,226 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,226 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:03,226 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:03,227 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:03,227 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:03,227 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,227 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:02:03,227 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:02:03,227 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:03,227 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:03,227 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:03,227 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:03,227 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. started... 2023-02-13 22:02:03,227 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d916f52616da598e54eb3efe0a77314e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:02:03,236 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,238 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,238 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,239 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:03,239 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,239 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:03,239 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/6ad747cf43f24b7dbf174b4d4237b50b 2023-02-13 22:02:03,249 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/d1ca54ab73c244c9ad3aa6b4cacac217 2023-02-13 22:02:03,253 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/6ad747cf43f24b7dbf174b4d4237b50b as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b 2023-02-13 22:02:03,257 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:02:03,258 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/d1ca54ab73c244c9ad3aa6b4cacac217 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217 2023-02-13 22:02:03,263 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 22:02:03,264 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d916f52616da598e54eb3efe0a77314e in 37ms, sequenceid=58, compaction requested=true 2023-02-13 22:02:03,264 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d916f52616da598e54eb3efe0a77314e: 2023-02-13 22:02:03,264 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:03,264 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:02:03,264 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:02:03,264 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,264 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:03,264 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:03,266 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,266 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:03,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:03,266 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,266 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:03,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:03,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:03,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:03,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 22:02:03,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6fd47bef[Count = 0] remaining members to acquire global barrier 2023-02-13 22:02:03,269 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 22:02:03,269 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,270 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,270 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,270 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,270 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,270 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:02:03,270 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:03,270 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:03,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,270 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:02:03,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,270 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:03,270 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:03,270 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:02:03,270 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:03,271 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:03,271 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:02:03,271 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:03,271 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,271 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:03,271 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,271 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:03,271 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:03,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:03,272 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:03,272 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:03,272 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,272 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:03,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:03,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:03,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:03,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:02:03,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 22:02:03,275 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 22:02:03,275 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:02:03,275 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:02:03,275 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 22:02:03,275 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:02:03,276 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,276 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,277 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,277 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,276 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,276 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,277 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,277 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:03,278 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,279 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,281 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:03,282 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,282 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,288 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,289 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 22:02:03,289 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:02:03,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,289 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,289 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,290 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,290 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,290 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,290 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,390 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:02:03,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:02:03,400 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-13 22:02:03,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 22:02:03,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 22:02:03,401 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 22:02:03,401 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,401 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 22:02:03,401 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 22:02:03,401 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,401 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,402 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,402 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,402 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,402 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,402 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,402 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 22:02:03,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:03,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:03,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:03,404 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 22:02:03,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:03,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:03,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,42047,1676325615467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33193,1676325615429' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:03,404 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 22:02:03,405 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 22:02:03,405 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:03,405 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. started... 2023-02-13 22:02:03,405 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d916f52616da598e54eb3efe0a77314e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 22:02:03,406 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,406 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,406 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,407 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:03,407 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,407 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:03,815 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/53420e13a69142208511d4f6fd4374e7 2023-02-13 22:02:03,835 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/1c75ef9bebf9404192cfe087b4b2a4e7 2023-02-13 22:02:03,840 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/53420e13a69142208511d4f6fd4374e7 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7 2023-02-13 22:02:03,846 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:02:03,847 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/1c75ef9bebf9404192cfe087b4b2a4e7 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7 2023-02-13 22:02:03,850 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 22:02:03,850 INFO [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d916f52616da598e54eb3efe0a77314e in 445ms, sequenceid=71, compaction requested=true 2023-02-13 22:02:03,851 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d916f52616da598e54eb3efe0a77314e: 2023-02-13 22:02:03,851 DEBUG [rs(jenkins-hbase20.apache.org,37979,1676325615388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:03,851 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 22:02:03,851 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 22:02:03,851 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,851 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 22:02:03,851 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 22:02:03,852 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,852 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:03,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:03,852 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,852 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 22:02:03,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:03,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:03,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:03,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37979,1676325615388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 22:02:03,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c8ca626[Count = 0] remaining members to acquire global barrier 2023-02-13 22:02:03,854 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 22:02:03,854 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,854 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,854 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,854 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,854 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:03,854 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:03,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,854 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:03,855 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:03,854 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,855 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 22:02:03,854 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33193,1676325615429' in zk 2023-02-13 22:02:03,855 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37979,1676325615388' in zk 2023-02-13 22:02:03,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,855 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 22:02:03,855 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 22:02:03,855 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,42047,1676325615467' in zk 2023-02-13 22:02:03,855 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:03,855 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,855 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,855 DEBUG [member: 'jenkins-hbase20.apache.org,37979,1676325615388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:03,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:03,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:03,856 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:03,856 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,856 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 22:02:03,856 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,856 DEBUG [member: 'jenkins-hbase20.apache.org,33193,1676325615429' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:03,856 DEBUG [member: 'jenkins-hbase20.apache.org,42047,1676325615467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 22:02:03,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:03,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:03,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:03,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,37979,1676325615388': 2023-02-13 22:02:03,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37979,1676325615388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 22:02:03,858 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 22:02:03,858 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 22:02:03,858 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 22:02:03,858 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 22:02:03,858 INFO [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 22:02:03,859 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,859 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,859 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,859 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,859 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 22:02:03,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 22:02:03,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,859 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,859 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 22:02:03,860 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,861 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 22:02:03,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,861 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 22:02:03,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 22:02:03,863 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,863 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,863 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,880 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,880 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,880 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,880 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 22:02:03,880 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 22:02:03,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 22:02:03,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,880 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,880 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,880 DEBUG [(jenkins-hbase20.apache.org,36939,1676325614416)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 22:02:03,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 22:02:03,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,880 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 22:02:03,880 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,881 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,881 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 22:02:03,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:02:03,881 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 22:02:03,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,881 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 22:02:03,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:02:03,881 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 22:02:03,882 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:03,882 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,882 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:03,882 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,882 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 22:02:03,882 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 22:02:03,981 DEBUG [Listener at localhost.localdomain/42295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 22:02:03,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 22:02:03,996 DEBUG [Listener at localhost.localdomain/42295] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:03,997 DEBUG [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=d916f52616da598e54eb3efe0a77314e, daughterA=6b2d814b16f79b632f22df332505b176, daughterB=7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:03,999 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=d916f52616da598e54eb3efe0a77314e, daughterA=6b2d814b16f79b632f22df332505b176, daughterB=7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:03,999 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=d916f52616da598e54eb3efe0a77314e, daughterA=6b2d814b16f79b632f22df332505b176, daughterB=7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:03,999 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=d916f52616da598e54eb3efe0a77314e, daughterA=6b2d814b16f79b632f22df332505b176, daughterB=7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:04,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=d916f52616da598e54eb3efe0a77314e, UNASSIGN}] 2023-02-13 22:02:04,002 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=d916f52616da598e54eb3efe0a77314e, UNASSIGN 2023-02-13 22:02:04,003 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=d916f52616da598e54eb3efe0a77314e, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:04,003 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325724003"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325724003"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325724003"}]},"ts":"1676325724003"} 2023-02-13 22:02:04,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure d916f52616da598e54eb3efe0a77314e, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:02:04,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing d916f52616da598e54eb3efe0a77314e, disabling compactions & flushes 2023-02-13 22:02:04,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:04,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:04,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. after waiting 0 ms 2023-02-13 22:02:04,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:04,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing d916f52616da598e54eb3efe0a77314e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 22:02:04,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/8f7d48830a5b439cb02dff04d4e058a6 2023-02-13 22:02:04,189 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8 2023-02-13 22:02:04,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf1/8f7d48830a5b439cb02dff04d4e058a6 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6 2023-02-13 22:02:04,200 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:02:04,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/.tmp/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8 2023-02-13 22:02:04,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 22:02:04,205 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d916f52616da598e54eb3efe0a77314e in 47ms, sequenceid=82, compaction requested=true 2023-02-13 22:02:04,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 22:02:04,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:02:04,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. 2023-02-13 22:02:04,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for d916f52616da598e54eb3efe0a77314e: Waiting for close lock at 1676325724157Running coprocessor pre-close hooks at 1676325724157Disabling compacts and flushes for region at 1676325724157Disabling writes for close at 1676325724158 (+1 ms)Obtaining lock to block concurrent updates at 1676325724158Preparing flush snapshotting stores in d916f52616da598e54eb3efe0a77314e at 1676325724158Finished memstore snapshotting testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676325724159 (+1 ms)Flushing stores of testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. at 1676325724160 (+1 ms)Flushing d916f52616da598e54eb3efe0a77314e/cf1: creating writer at 1676325724162 (+2 ms)Flushing d916f52616da598e54eb3efe0a77314e/cf1: appending metadata at 1676325724165 (+3 ms)Flushing d916f52616da598e54eb3efe0a77314e/cf1: closing flushed file at 1676325724165Flushing d916f52616da598e54eb3efe0a77314e/cf2: creating writer at 1676325724179 (+14 ms)Flushing d916f52616da598e54eb3efe0a77314e/cf2: appending metadata at 1676325724181 (+2 ms)Flushing d916f52616da598e54eb3efe0a77314e/cf2: closing flushed file at 1676325724181Flushing d916f52616da598e54eb3efe0a77314e/cf1: reopening flushed file at 1676325724195 (+14 ms)Flushing d916f52616da598e54eb3efe0a77314e/cf2: reopening flushed file at 1676325724202 (+7 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d916f52616da598e54eb3efe0a77314e in 47ms, sequenceid=82, compaction requested=true at 1676325724205 (+3 ms)Writing region close event to WAL at 1676325724207 (+2 ms)Running coprocessor post-close hooks at 1676325724209 (+2 ms)Closed at 1676325724209 2023-02-13 22:02:04,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,210 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=d916f52616da598e54eb3efe0a77314e, regionState=CLOSED 2023-02-13 22:02:04,210 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325724210"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325724210"}]},"ts":"1676325724210"} 2023-02-13 22:02:04,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-13 22:02:04,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure d916f52616da598e54eb3efe0a77314e, server=jenkins-hbase20.apache.org,37979,1676325615388 in 207 msec 2023-02-13 22:02:04,213 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-13 22:02:04,213 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=d916f52616da598e54eb3efe0a77314e, UNASSIGN in 211 msec 2023-02-13 22:02:04,223 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=d916f52616da598e54eb3efe0a77314e, threads=14 2023-02-13 22:02:04,224 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,224 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,224 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,224 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,224 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,224 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,224 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,224 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,224 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,224 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,225 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,225 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,225 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,225 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,303 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,303 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,311 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,311 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,313 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,316 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,316 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,317 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,319 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,320 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,322 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,696 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,702 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,708 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10 for region: d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:04,709 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region d916f52616da598e54eb3efe0a77314e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 22:02:05,170 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:02:05,172 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 22:02:05,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676325725173"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676325725173"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676325725173"}]},"ts":"1676325725173"} 2023-02-13 22:02:05,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325725173"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325725173"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325725173"}]},"ts":"1676325725173"} 2023-02-13 22:02:05,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325725173"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325725173"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325725173"}]},"ts":"1676325725173"} 2023-02-13 22:02:05,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=6b2d814b16f79b632f22df332505b176, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=7b9679ed4fea58b48be27b8fabf68208, ASSIGN}] 2023-02-13 22:02:05,177 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=7b9679ed4fea58b48be27b8fabf68208, ASSIGN 2023-02-13 22:02:05,178 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=7b9679ed4fea58b48be27b8fabf68208, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,37979,1676325615388; forceNewPlan=false, retain=false 2023-02-13 22:02:05,179 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=6b2d814b16f79b632f22df332505b176, ASSIGN 2023-02-13 22:02:05,179 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=6b2d814b16f79b632f22df332505b176, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,37979,1676325615388; forceNewPlan=false, retain=false 2023-02-13 22:02:05,185 ERROR [BP-109365581-148.251.75.209-1676325611364 heartbeating to localhost.localdomain/127.0.0.1:37463] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-109365581-148.251.75.209-1676325611364 (Datanode Uuid 3f4de7bc-de25-42a6-88c4-90335eaeb4a2) service to localhost.localdomain/127.0.0.1:37463 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 22:02:05,328 INFO [jenkins-hbase20:36939] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 22:02:05,332 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=6b2d814b16f79b632f22df332505b176, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:05,332 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325725332"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325725332"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325725332"}]},"ts":"1676325725332"} 2023-02-13 22:02:05,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 6b2d814b16f79b632f22df332505b176, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:02:05,341 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=7b9679ed4fea58b48be27b8fabf68208, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:05,341 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325725341"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325725341"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325725341"}]},"ts":"1676325725341"} 2023-02-13 22:02:05,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 7b9679ed4fea58b48be27b8fabf68208, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:02:05,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:05,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6b2d814b16f79b632f22df332505b176, NAME => 'testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 22:02:05,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:02:05,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:05,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:02:05,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:05,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:05,500 INFO [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:05,502 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1 2023-02-13 22:02:05,502 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1 2023-02-13 22:02:05,503 INFO [StoreOpener-6b2d814b16f79b632f22df332505b176-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b2d814b16f79b632f22df332505b176 columnFamilyName cf1 2023-02-13 22:02:05,517 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068-bottom 2023-02-13 22:02:05,523 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114-bottom 2023-02-13 22:02:05,528 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c-bottom 2023-02-13 22:02:05,533 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7-bottom 2023-02-13 22:02:05,538 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b-bottom 2023-02-13 22:02:05,542 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6-bottom 2023-02-13 22:02:05,545 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33-bottom 2023-02-13 22:02:05,546 INFO [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(310): Store=6b2d814b16f79b632f22df332505b176/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:02:05,546 INFO [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:05,546 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2 2023-02-13 22:02:05,547 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2 2023-02-13 22:02:05,547 INFO [StoreOpener-6b2d814b16f79b632f22df332505b176-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b2d814b16f79b632f22df332505b176 columnFamilyName cf2 2023-02-13 22:02:05,560 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7-bottom 2023-02-13 22:02:05,564 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10-bottom 2023-02-13 22:02:05,568 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209-bottom 2023-02-13 22:02:05,572 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8-bottom 2023-02-13 22:02:05,577 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a-bottom 2023-02-13 22:02:05,581 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5-bottom 2023-02-13 22:02:05,586 DEBUG [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217-bottom 2023-02-13 22:02:05,586 INFO [StoreOpener-6b2d814b16f79b632f22df332505b176-1] regionserver.HStore(310): Store=6b2d814b16f79b632f22df332505b176/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:02:05,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:05,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:05,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:02:05,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:05,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 6b2d814b16f79b632f22df332505b176; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69155321, jitterRate=0.030494585633277893}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:02:05,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 6b2d814b16f79b632f22df332505b176: Running coprocessor pre-open hook at 1676325725498Writing region info on filesystem at 1676325725498Initializing all the Stores at 1676325725500 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325725500Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325725500Cleaning up temporary data from old regions at 1676325725588 (+88 ms)Cleaning up detritus from prior splits at 1676325725589 (+1 ms)Running coprocessor post-open hooks at 1676325725591 (+2 ms)Region opened successfully at 1676325725591 2023-02-13 22:02:05,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176., pid=149, masterSystemTime=1676325725489 2023-02-13 22:02:05,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:02:05,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:02:05,593 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:02:05,593 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:02:05,593 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:05,593 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:05,593 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1912): 6b2d814b16f79b632f22df332505b176/cf2 is initiating minor compaction (all files) 2023-02-13 22:02:05,593 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1912): 6b2d814b16f79b632f22df332505b176/cf1 is initiating minor compaction (all files) 2023-02-13 22:02:05,593 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6b2d814b16f79b632f22df332505b176/cf2 in testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:05,593 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6b2d814b16f79b632f22df332505b176/cf1 in testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:05,593 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8-bottom] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/.tmp, totalSize=34.9 K 2023-02-13 22:02:05,594 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6-bottom] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/.tmp, totalSize=34.9 K 2023-02-13 22:02:05,594 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325722478 2023-02-13 22:02:05,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:05,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:05,594 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676325722478 2023-02-13 22:02:05,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:05,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=6b2d814b16f79b632f22df332505b176, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:05,595 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325722669 2023-02-13 22:02:05,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7b9679ed4fea58b48be27b8fabf68208, NAME => 'testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 22:02:05,595 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325725595"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325725595"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325725595"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325725595"}]},"ts":"1676325725595"} 2023-02-13 22:02:05,595 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676325722669 2023-02-13 22:02:05,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 22:02:05,595 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325722847 2023-02-13 22:02:05,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:05,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:02:05,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:05,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:05,596 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676325722847 2023-02-13 22:02:05,596 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325723025 2023-02-13 22:02:05,597 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325723205 2023-02-13 22:02:05,597 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676325723025 2023-02-13 22:02:05,597 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325723391 2023-02-13 22:02:05,598 INFO [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:05,598 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676325723205 2023-02-13 22:02:05,598 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325723983 2023-02-13 22:02:05,598 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676325723391 2023-02-13 22:02:05,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-13 22:02:05,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 6b2d814b16f79b632f22df332505b176, server=jenkins-hbase20.apache.org,37979,1676325615388 in 261 msec 2023-02-13 22:02:05,599 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676325723983 2023-02-13 22:02:05,601 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1 2023-02-13 22:02:05,601 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1 2023-02-13 22:02:05,601 INFO [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b9679ed4fea58b48be27b8fabf68208 columnFamilyName cf1 2023-02-13 22:02:05,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=6b2d814b16f79b632f22df332505b176, ASSIGN in 423 msec 2023-02-13 22:02:05,608 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] throttle.PressureAwareThroughputController(145): 6b2d814b16f79b632f22df332505b176#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:02:05,608 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6b2d814b16f79b632f22df332505b176#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:02:05,612 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068-top 2023-02-13 22:02:05,614 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114-top 2023-02-13 22:02:05,616 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/.tmp/cf1/601749982c734cb79fb47b5ce36a05c1 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/601749982c734cb79fb47b5ce36a05c1 2023-02-13 22:02:05,616 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/.tmp/cf2/663402c66fd245d5915874ead0593d0c as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/663402c66fd245d5915874ead0593d0c 2023-02-13 22:02:05,616 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c-top 2023-02-13 22:02:05,619 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7-top 2023-02-13 22:02:05,620 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6b2d814b16f79b632f22df332505b176/cf1 of 6b2d814b16f79b632f22df332505b176 into 601749982c734cb79fb47b5ce36a05c1(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:02:05,620 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6b2d814b16f79b632f22df332505b176: 2023-02-13 22:02:05,620 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176., storeName=6b2d814b16f79b632f22df332505b176/cf1, priority=9, startTime=1676325725592; duration=0sec 2023-02-13 22:02:05,620 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:02:05,620 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6b2d814b16f79b632f22df332505b176/cf2 of 6b2d814b16f79b632f22df332505b176 into 663402c66fd245d5915874ead0593d0c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:02:05,620 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6b2d814b16f79b632f22df332505b176: 2023-02-13 22:02:05,620 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176., storeName=6b2d814b16f79b632f22df332505b176/cf2, priority=9, startTime=1676325725592; duration=0sec 2023-02-13 22:02:05,620 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:02:05,622 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b-top 2023-02-13 22:02:05,625 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6-top 2023-02-13 22:02:05,628 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33-top 2023-02-13 22:02:05,629 INFO [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(310): Store=7b9679ed4fea58b48be27b8fabf68208/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:02:05,629 INFO [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:05,629 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2 2023-02-13 22:02:05,630 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2 2023-02-13 22:02:05,630 INFO [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b9679ed4fea58b48be27b8fabf68208 columnFamilyName cf2 2023-02-13 22:02:05,641 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7-top 2023-02-13 22:02:05,643 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10-top 2023-02-13 22:02:05,646 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209-top 2023-02-13 22:02:05,649 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8-top 2023-02-13 22:02:05,651 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a-top 2023-02-13 22:02:05,654 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5-top 2023-02-13 22:02:05,657 DEBUG [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217-top 2023-02-13 22:02:05,657 INFO [StoreOpener-7b9679ed4fea58b48be27b8fabf68208-1] regionserver.HStore(310): Store=7b9679ed4fea58b48be27b8fabf68208/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:02:05,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:05,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:05,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 22:02:05,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:05,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 7b9679ed4fea58b48be27b8fabf68208; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69188459, jitterRate=0.030988380312919617}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 22:02:05,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 7b9679ed4fea58b48be27b8fabf68208: Running coprocessor pre-open hook at 1676325725596Writing region info on filesystem at 1676325725596Initializing all the Stores at 1676325725597 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325725597Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325725597Cleaning up temporary data from old regions at 1676325725659 (+62 ms)Cleaning up detritus from prior splits at 1676325725659Running coprocessor post-open hooks at 1676325725661 (+2 ms)Region opened successfully at 1676325725661 2023-02-13 22:02:05,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208., pid=150, masterSystemTime=1676325725489 2023-02-13 22:02:05,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:02:05,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 22:02:05,662 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:02:05,662 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 22:02:05,662 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:05,662 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:05,663 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1912): 7b9679ed4fea58b48be27b8fabf68208/cf2 is initiating minor compaction (all files) 2023-02-13 22:02:05,663 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1912): 7b9679ed4fea58b48be27b8fabf68208/cf1 is initiating minor compaction (all files) 2023-02-13 22:02:05,663 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 7b9679ed4fea58b48be27b8fabf68208/cf2 in testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:05,663 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7b9679ed4fea58b48be27b8fabf68208/cf1 in testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:05,663 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8-top] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/.tmp, totalSize=34.9 K 2023-02-13 22:02:05,663 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6-top] into tmpdir=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/.tmp, totalSize=34.9 K 2023-02-13 22:02:05,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:05,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:05,664 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676325722478 2023-02-13 22:02:05,664 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676325722478 2023-02-13 22:02:05,664 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=7b9679ed4fea58b48be27b8fabf68208, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:05,664 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325725664"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325725664"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325725664"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325725664"}]},"ts":"1676325725664"} 2023-02-13 22:02:05,664 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676325722669 2023-02-13 22:02:05,664 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676325722669 2023-02-13 22:02:05,664 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676325722847 2023-02-13 22:02:05,665 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676325722847 2023-02-13 22:02:05,665 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676325723025 2023-02-13 22:02:05,665 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676325723025 2023-02-13 22:02:05,665 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676325723205 2023-02-13 22:02:05,665 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676325723205 2023-02-13 22:02:05,665 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676325723391 2023-02-13 22:02:05,666 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676325723391 2023-02-13 22:02:05,666 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] compactions.Compactor(207): Compacting 938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676325723983 2023-02-13 22:02:05,666 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] compactions.Compactor(207): Compacting 8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676325723983 2023-02-13 22:02:05,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-13 22:02:05,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 7b9679ed4fea58b48be27b8fabf68208, server=jenkins-hbase20.apache.org,37979,1676325615388 in 322 msec 2023-02-13 22:02:05,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-13 22:02:05,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=7b9679ed4fea58b48be27b8fabf68208, ASSIGN in 490 msec 2023-02-13 22:02:05,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=d916f52616da598e54eb3efe0a77314e, daughterA=6b2d814b16f79b632f22df332505b176, daughterB=7b9679ed4fea58b48be27b8fabf68208 in 1.6730 sec 2023-02-13 22:02:05,676 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] throttle.PressureAwareThroughputController(145): 7b9679ed4fea58b48be27b8fabf68208#cf2#compaction#144 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:02:05,676 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7b9679ed4fea58b48be27b8fabf68208#cf1#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 22:02:05,688 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/.tmp/cf2/8b573d2594aa4782b55efc4aefe76362 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/8b573d2594aa4782b55efc4aefe76362 2023-02-13 22:02:05,692 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7b9679ed4fea58b48be27b8fabf68208/cf2 of 7b9679ed4fea58b48be27b8fabf68208 into 8b573d2594aa4782b55efc4aefe76362(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:02:05,692 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7b9679ed4fea58b48be27b8fabf68208: 2023-02-13 22:02:05,692 INFO [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208., storeName=7b9679ed4fea58b48be27b8fabf68208/cf2, priority=9, startTime=1676325725662; duration=0sec 2023-02-13 22:02:05,692 DEBUG [RS:0;jenkins-hbase20:37979-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:02:05,693 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/.tmp/cf1/64abc1716a5e4c35a2ee4f6983fb7b00 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/64abc1716a5e4c35a2ee4f6983fb7b00 2023-02-13 22:02:05,696 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7b9679ed4fea58b48be27b8fabf68208/cf1 of 7b9679ed4fea58b48be27b8fabf68208 into 64abc1716a5e4c35a2ee4f6983fb7b00(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 22:02:05,696 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7b9679ed4fea58b48be27b8fabf68208: 2023-02-13 22:02:05,696 INFO [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208., storeName=7b9679ed4fea58b48be27b8fabf68208/cf1, priority=9, startTime=1676325725662; duration=0sec 2023-02-13 22:02:05,697 DEBUG [RS:0;jenkins-hbase20:37979-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 22:02:05,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37979] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 148.251.75.209:50934 deadline: 1676325785749, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e. is not online on jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:05,865 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 22:02:05,876 INFO [Listener at localhost.localdomain/42295] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 22:02:05,878 DEBUG [Listener at localhost.localdomain/42295] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 22:02:05,888 WARN [Listener at localhost.localdomain/42295] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 22:02:05,889 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-13 22:02:05,889 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitTableRegion 2023-02-13 22:02:05,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-13 22:02:05,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 22:02:05,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325725892"}]},"ts":"1676325725892"} 2023-02-13 22:02:05,893 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-13 22:02:05,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 22:02:06,014 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-13 22:02:06,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=6b2d814b16f79b632f22df332505b176, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=7b9679ed4fea58b48be27b8fabf68208, UNASSIGN}] 2023-02-13 22:02:06,018 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=7b9679ed4fea58b48be27b8fabf68208, UNASSIGN 2023-02-13 22:02:06,019 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=7b9679ed4fea58b48be27b8fabf68208, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:06,020 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325726019"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325726019"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325726019"}]},"ts":"1676325726019"} 2023-02-13 22:02:06,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 7b9679ed4fea58b48be27b8fabf68208, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:02:06,026 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=6b2d814b16f79b632f22df332505b176, UNASSIGN 2023-02-13 22:02:06,027 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=6b2d814b16f79b632f22df332505b176, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:06,027 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325726027"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325726027"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325726027"}]},"ts":"1676325726027"} 2023-02-13 22:02:06,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 6b2d814b16f79b632f22df332505b176, server=jenkins-hbase20.apache.org,37979,1676325615388}] 2023-02-13 22:02:06,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:06,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 6b2d814b16f79b632f22df332505b176, disabling compactions & flushes 2023-02-13 22:02:06,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:06,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:06,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. after waiting 0 ms 2023-02-13 22:02:06,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:06,180 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6-bottom] to archive 2023-02-13 22:02:06,182 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:02:06,187 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,191 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,192 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,194 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 22:02:06,196 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,198 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,200 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,202 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7-bottom, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8-bottom] to archive 2023-02-13 22:02:06,203 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:02:06,205 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,207 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,209 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,210 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,212 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,214 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,216 DEBUG [StoreCloser-testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 22:02:06,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:02:06,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176. 2023-02-13 22:02:06,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 6b2d814b16f79b632f22df332505b176: Waiting for close lock at 1676325726177Running coprocessor pre-close hooks at 1676325726177Disabling compacts and flushes for region at 1676325726177Disabling writes for close at 1676325726178 (+1 ms)Writing region close event to WAL at 1676325726218 (+40 ms)Running coprocessor post-close hooks at 1676325726223 (+5 ms)Closed at 1676325726223 2023-02-13 22:02:06,225 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:06,225 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:06,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 7b9679ed4fea58b48be27b8fabf68208, disabling compactions & flushes 2023-02-13 22:02:06,225 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:06,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:06,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. after waiting 0 ms 2023-02-13 22:02:06,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:06,225 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=6b2d814b16f79b632f22df332505b176, regionState=CLOSED 2023-02-13 22:02:06,225 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325726225"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325726225"}]},"ts":"1676325726225"} 2023-02-13 22:02:06,227 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6-top] to archive 2023-02-13 22:02:06,228 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:02:06,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-13 22:02:06,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 6b2d814b16f79b632f22df332505b176, server=jenkins-hbase20.apache.org,37979,1676325615388 in 199 msec 2023-02-13 22:02:06,230 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/cbbec0e884ef47bfb33c98cac8b97b33.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=6b2d814b16f79b632f22df332505b176, UNASSIGN in 213 msec 2023-02-13 22:02:06,231 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/0c07e9e45e564156b9aeca4d60a7e068.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,233 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/48237147796140ddbd944793d35ae114.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,234 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,235 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/6ad747cf43f24b7dbf174b4d4237b50b.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,236 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/53420e13a69142208511d4f6fd4374e7.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,237 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/8f7d48830a5b439cb02dff04d4e058a6.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,239 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7-top, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e->hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8-top] to archive 2023-02-13 22:02:06,240 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 22:02:06,241 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/c317f00b85434aa48d57983ab25c22a5.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,241 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/73af04a2269445a5aa92ce2079b92209.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,242 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/3d4fd1ca001041ef84257eb61351ad10.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,243 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/94e4e89845964aa199e34d446dfbee8a.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,244 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/d1ca54ab73c244c9ad3aa6b4cacac217.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,244 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/1c75ef9bebf9404192cfe087b4b2a4e7.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,245 DEBUG [StoreCloser-testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8.d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 22:02:06,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:02:06,248 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208. 2023-02-13 22:02:06,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 7b9679ed4fea58b48be27b8fabf68208: Waiting for close lock at 1676325726225Running coprocessor pre-close hooks at 1676325726225Disabling compacts and flushes for region at 1676325726225Disabling writes for close at 1676325726225Writing region close event to WAL at 1676325726246 (+21 ms)Running coprocessor post-close hooks at 1676325726248 (+2 ms)Closed at 1676325726248 2023-02-13 22:02:06,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:06,249 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=7b9679ed4fea58b48be27b8fabf68208, regionState=CLOSED 2023-02-13 22:02:06,249 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676325726249"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325726249"}]},"ts":"1676325726249"} 2023-02-13 22:02:06,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-13 22:02:06,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 7b9679ed4fea58b48be27b8fabf68208, server=jenkins-hbase20.apache.org,37979,1676325615388 in 227 msec 2023-02-13 22:02:06,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-02-13 22:02:06,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=7b9679ed4fea58b48be27b8fabf68208, UNASSIGN in 235 msec 2023-02-13 22:02:06,252 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325726252"}]},"ts":"1676325726252"} 2023-02-13 22:02:06,253 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-13 22:02:06,254 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-13 22:02:06,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 364 msec 2023-02-13 22:02:06,374 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 22:02:06,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 22:02:06,497 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-13 22:02:06,498 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitTableRegion 2023-02-13 22:02:06,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 22:02:06,501 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 22:02:06,501 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 22:02:06,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 22:02:06,507 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,507 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:06,507 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:06,510 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/recovered.edits] 2023-02-13 22:02:06,510 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/recovered.edits] 2023-02-13 22:02:06,511 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2, FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/recovered.edits] 2023-02-13 22:02:06,514 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/64abc1716a5e4c35a2ee4f6983fb7b00 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf1/64abc1716a5e4c35a2ee4f6983fb7b00 2023-02-13 22:02:06,514 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/601749982c734cb79fb47b5ce36a05c1 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf1/601749982c734cb79fb47b5ce36a05c1 2023-02-13 22:02:06,515 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/0c07e9e45e564156b9aeca4d60a7e068 2023-02-13 22:02:06,517 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/48237147796140ddbd944793d35ae114 2023-02-13 22:02:06,517 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/663402c66fd245d5915874ead0593d0c to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/cf2/663402c66fd245d5915874ead0593d0c 2023-02-13 22:02:06,518 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/8b573d2594aa4782b55efc4aefe76362 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/cf2/8b573d2594aa4782b55efc4aefe76362 2023-02-13 22:02:06,518 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/4ed0d8ce21af49b7a537b7ba2c03cb9c 2023-02-13 22:02:06,520 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/53420e13a69142208511d4f6fd4374e7 2023-02-13 22:02:06,521 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/recovered.edits/90.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176/recovered.edits/90.seqid 2023-02-13 22:02:06,521 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/recovered.edits/90.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208/recovered.edits/90.seqid 2023-02-13 22:02:06,521 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/6b2d814b16f79b632f22df332505b176 2023-02-13 22:02:06,522 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/6ad747cf43f24b7dbf174b4d4237b50b 2023-02-13 22:02:06,522 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/7b9679ed4fea58b48be27b8fabf68208 2023-02-13 22:02:06,523 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/8f7d48830a5b439cb02dff04d4e058a6 2023-02-13 22:02:06,524 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf1/cbbec0e884ef47bfb33c98cac8b97b33 2023-02-13 22:02:06,527 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/1c75ef9bebf9404192cfe087b4b2a4e7 2023-02-13 22:02:06,528 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/3d4fd1ca001041ef84257eb61351ad10 2023-02-13 22:02:06,530 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/73af04a2269445a5aa92ce2079b92209 2023-02-13 22:02:06,531 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/938f5f2ab24f4d8eb005c3b8ac6c3be8 2023-02-13 22:02:06,532 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/94e4e89845964aa199e34d446dfbee8a 2023-02-13 22:02:06,534 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/c317f00b85434aa48d57983ab25c22a5 2023-02-13 22:02:06,535 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217 to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/cf2/d1ca54ab73c244c9ad3aa6b4cacac217 2023-02-13 22:02:06,539 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/recovered.edits/85.seqid to hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/archive/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e/recovered.edits/85.seqid 2023-02-13 22:02:06,539 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/.tmp/data/default/testSplitTableRegion/d916f52616da598e54eb3efe0a77314e 2023-02-13 22:02:06,539 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 22:02:06,542 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 22:02:06,544 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-13 22:02:06,547 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-13 22:02:06,548 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 22:02:06,548 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-13 22:02:06,549 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325726548"}]},"ts":"9223372036854775807"} 2023-02-13 22:02:06,549 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325726548"}]},"ts":"9223372036854775807"} 2023-02-13 22:02:06,549 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676325726548"}]},"ts":"9223372036854775807"} 2023-02-13 22:02:06,551 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 22:02:06,551 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d916f52616da598e54eb3efe0a77314e, NAME => 'testSplitTableRegion,,1676325721972.d916f52616da598e54eb3efe0a77314e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6b2d814b16f79b632f22df332505b176, NAME => 'testSplitTableRegion,,1676325723996.6b2d814b16f79b632f22df332505b176.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 7b9679ed4fea58b48be27b8fabf68208, NAME => 'testSplitTableRegion,41,1676325723996.7b9679ed4fea58b48be27b8fabf68208.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 22:02:06,551 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-13 22:02:06,552 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676325726552"}]},"ts":"9223372036854775807"} 2023-02-13 22:02:06,553 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-13 22:02:06,554 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 22:02:06,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 56 msec 2023-02-13 22:02:06,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36939] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 22:02:06,603 INFO [Listener at localhost.localdomain/42295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-13 22:02:06,614 INFO [Listener at localhost.localdomain/42295] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376 (was 375) - Thread LEAK? -, OpenFileDescriptor=835 (was 833) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=90 (was 98), ProcessCount=162 (was 162), AvailableMemoryMB=7598 (was 7602) 2023-02-13 22:02:06,614 INFO [Listener at localhost.localdomain/42295] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 22:02:06,617 INFO [Listener at localhost.localdomain/42295] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 22:02:06,617 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1753ea84 to 127.0.0.1:63985 2023-02-13 22:02:06,617 DEBUG [Listener at localhost.localdomain/42295] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:02:06,618 DEBUG [Listener at localhost.localdomain/42295] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 22:02:06,618 DEBUG [Listener at localhost.localdomain/42295] util.JVMClusterUtil(257): Found active master hash=831448046, stopped=false 2023-02-13 22:02:06,618 INFO [Listener at localhost.localdomain/42295] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase20.apache.org,36939,1676325614416 2023-02-13 22:02:06,619 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 22:02:06,619 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 22:02:06,619 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 22:02:06,619 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 22:02:06,619 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:02:06,619 INFO [Listener at localhost.localdomain/42295] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:02:06,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:02:06,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:02:06,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:02:06,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:02:06,620 DEBUG [Listener at localhost.localdomain/42295] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2ad34793 to 127.0.0.1:63985 2023-02-13 22:02:06,620 DEBUG [Listener at localhost.localdomain/42295] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:02:06,620 INFO [Listener at localhost.localdomain/42295] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,37979,1676325615388' ***** 2023-02-13 22:02:06,620 INFO [Listener at localhost.localdomain/42295] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 22:02:06,620 INFO [Listener at localhost.localdomain/42295] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,33193,1676325615429' ***** 2023-02-13 22:02:06,620 INFO [Listener at localhost.localdomain/42295] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 22:02:06,621 INFO [RS:0;jenkins-hbase20:37979] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 22:02:06,621 INFO [Listener at localhost.localdomain/42295] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,42047,1676325615467' ***** 2023-02-13 22:02:06,621 INFO [RS:1;jenkins-hbase20:33193] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 22:02:06,621 INFO [RS:0;jenkins-hbase20:37979] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 22:02:06,621 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 22:02:06,621 INFO [RS:1;jenkins-hbase20:33193] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 22:02:06,621 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 22:02:06,621 INFO [RS:1;jenkins-hbase20:33193] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 22:02:06,621 INFO [Listener at localhost.localdomain/42295] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 22:02:06,621 INFO [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:06,621 INFO [RS:0;jenkins-hbase20:37979] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 22:02:06,621 DEBUG [RS:1;jenkins-hbase20:33193] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x78dcafe0 to 127.0.0.1:63985 2023-02-13 22:02:06,621 INFO [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:06,622 DEBUG [RS:0;jenkins-hbase20:37979] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28fb1460 to 127.0.0.1:63985 2023-02-13 22:02:06,621 INFO [RS:2;jenkins-hbase20:42047] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 22:02:06,622 DEBUG [RS:0;jenkins-hbase20:37979] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:02:06,621 DEBUG [RS:1;jenkins-hbase20:33193] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:02:06,622 INFO [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,37979,1676325615388; all regions closed. 2023-02-13 22:02:06,622 INFO [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,33193,1676325615429; all regions closed. 2023-02-13 22:02:06,622 INFO [RS:2;jenkins-hbase20:42047] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 22:02:06,622 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 22:02:06,622 INFO [RS:2;jenkins-hbase20:42047] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 22:02:06,622 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(3304): Received CLOSE for 09ae03d8301e349bcc2ee47a4f5f6b0a 2023-02-13 22:02:06,622 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:06,622 DEBUG [RS:2;jenkins-hbase20:42047] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x25b1011f to 127.0.0.1:63985 2023-02-13 22:02:06,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 09ae03d8301e349bcc2ee47a4f5f6b0a, disabling compactions & flushes 2023-02-13 22:02:06,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:02:06,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:02:06,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. after waiting 0 ms 2023-02-13 22:02:06,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:02:06,623 DEBUG [RS:2;jenkins-hbase20:42047] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:02:06,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 09ae03d8301e349bcc2ee47a4f5f6b0a 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 22:02:06,623 INFO [RS:2;jenkins-hbase20:42047] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 22:02:06,623 INFO [RS:2;jenkins-hbase20:42047] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 22:02:06,623 INFO [RS:2;jenkins-hbase20:42047] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 22:02:06,623 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 22:02:06,623 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-13 22:02:06,623 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740, 09ae03d8301e349bcc2ee47a4f5f6b0a=hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a.} 2023-02-13 22:02:06,624 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 22:02:06,624 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 22:02:06,747 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 22:02:06,747 INFO [AsyncFSWAL-0-hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c-prefix:jenkins-hbase20.apache.org,42047,1676325615467] wal.AbstractFSWAL(1045): Slow sync cost: 123 ms, current pipeline: [DatanodeInfoWithStorage[127.0.0.1:33677,DS-0fe6d3b5-1055-4d3d-81a8-b68be6a7ca99,DISK], DatanodeInfoWithStorage[127.0.0.1:38805,DS-2c429e9a-bae9-41b0-9fb3-457ceda67ccf,DISK], DatanodeInfoWithStorage[127.0.0.1:45835,DS-aa2ba50c-f539-43c1-bafe-1a9171cfcd73,DISK]] 2023-02-13 22:02:06,747 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1505): Waiting on 09ae03d8301e349bcc2ee47a4f5f6b0a, 1588230740 2023-02-13 22:02:06,746 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:02:06,746 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:02:06,746 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:02:06,747 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 22:02:06,749 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 22:02:06,749 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-02-13 22:02:06,754 DEBUG [RS:1;jenkins-hbase20:33193] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/oldWALs 2023-02-13 22:02:06,754 DEBUG [RS:0;jenkins-hbase20:37979] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/oldWALs 2023-02-13 22:02:06,754 INFO [RS:1;jenkins-hbase20:33193] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C33193%2C1676325615429:(num 1676325616937) 2023-02-13 22:02:06,755 INFO [RS:0;jenkins-hbase20:37979] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C37979%2C1676325615388:(num 1676325616937) 2023-02-13 22:02:06,755 DEBUG [RS:0;jenkins-hbase20:37979] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:02:06,755 DEBUG [RS:1;jenkins-hbase20:33193] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:02:06,755 INFO [RS:0;jenkins-hbase20:37979] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:02:06,755 INFO [RS:1;jenkins-hbase20:33193] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:02:06,755 INFO [RS:1;jenkins-hbase20:33193] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 22:02:06,755 INFO [RS:1;jenkins-hbase20:33193] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 22:02:06,755 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:02:06,755 INFO [RS:1;jenkins-hbase20:33193] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 22:02:06,755 INFO [RS:1;jenkins-hbase20:33193] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 22:02:06,756 INFO [RS:0;jenkins-hbase20:37979] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 22:02:06,756 INFO [RS:1;jenkins-hbase20:33193] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:33193 2023-02-13 22:02:06,757 INFO [RS:0;jenkins-hbase20:37979] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 22:02:06,757 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:02:06,757 INFO [RS:0;jenkins-hbase20:37979] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 22:02:06,757 INFO [RS:0;jenkins-hbase20:37979] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 22:02:06,758 INFO [RS:0;jenkins-hbase20:37979] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:37979 2023-02-13 22:02:06,764 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:02:06,764 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:06,764 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:06,764 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,33193,1676325615429 2023-02-13 22:02:06,764 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:02:06,764 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:06,764 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:02:06,767 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@24fc3bca rejected from java.util.concurrent.ThreadPoolExecutor@20c3ab8d[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:06,767 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:06,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a/.tmp/info/75f2c64bcaaf438197d0e9a46d0c8ac6 2023-02-13 22:02:06,767 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@863018e rejected from java.util.concurrent.ThreadPoolExecutor@c5c845a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:06,767 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1109399c rejected from java.util.concurrent.ThreadPoolExecutor@20c3ab8d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:06,767 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:02:06,767 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6822600b rejected from java.util.concurrent.ThreadPoolExecutor@c5c845a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:06,767 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,37979,1676325615388 2023-02-13 22:02:06,767 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,33193,1676325615429] 2023-02-13 22:02:06,767 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@10cb56f5 rejected from java.util.concurrent.ThreadPoolExecutor@c5c845a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:06,768 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,33193,1676325615429; numProcessing=1 2023-02-13 22:02:06,768 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,33193,1676325615429 already deleted, retry=false 2023-02-13 22:02:06,768 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,33193,1676325615429 expired; onlineServers=2 2023-02-13 22:02:06,768 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,37979,1676325615388] 2023-02-13 22:02:06,768 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,37979,1676325615388; numProcessing=2 2023-02-13 22:02:06,769 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,37979,1676325615388 already deleted, retry=false 2023-02-13 22:02:06,769 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,37979,1676325615388 expired; onlineServers=1 2023-02-13 22:02:06,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a/.tmp/info/75f2c64bcaaf438197d0e9a46d0c8ac6 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a/info/75f2c64bcaaf438197d0e9a46d0c8ac6 2023-02-13 22:02:06,772 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/.tmp/info/e650a1c18cdb4b54a3833f259e5038b5 2023-02-13 22:02:06,774 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a/info/75f2c64bcaaf438197d0e9a46d0c8ac6, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:02:06,774 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e650a1c18cdb4b54a3833f259e5038b5 2023-02-13 22:02:06,774 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 09ae03d8301e349bcc2ee47a4f5f6b0a in 151ms, sequenceid=6, compaction requested=false 2023-02-13 22:02:06,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/namespace/09ae03d8301e349bcc2ee47a4f5f6b0a/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 22:02:06,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:02:06,778 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:02:06,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 09ae03d8301e349bcc2ee47a4f5f6b0a: Waiting for close lock at 1676325726623Running coprocessor pre-close hooks at 1676325726623Disabling compacts and flushes for region at 1676325726623Disabling writes for close at 1676325726623Obtaining lock to block concurrent updates at 1676325726623Preparing flush snapshotting stores in 09ae03d8301e349bcc2ee47a4f5f6b0a at 1676325726623Finished memstore snapshotting hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676325726623Flushing stores of hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. at 1676325726747 (+124 ms)Flushing 09ae03d8301e349bcc2ee47a4f5f6b0a/info: creating writer at 1676325726747Flushing 09ae03d8301e349bcc2ee47a4f5f6b0a/info: appending metadata at 1676325726753 (+6 ms)Flushing 09ae03d8301e349bcc2ee47a4f5f6b0a/info: closing flushed file at 1676325726753Flushing 09ae03d8301e349bcc2ee47a4f5f6b0a/info: reopening flushed file at 1676325726771 (+18 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 09ae03d8301e349bcc2ee47a4f5f6b0a in 151ms, sequenceid=6, compaction requested=false at 1676325726774 (+3 ms)Writing region close event to WAL at 1676325726775 (+1 ms)Running coprocessor post-close hooks at 1676325726778 (+3 ms)Closed at 1676325726778 2023-02-13 22:02:06,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676325617324.09ae03d8301e349bcc2ee47a4f5f6b0a. 2023-02-13 22:02:06,782 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/.tmp/rep_barrier/8c8fcdc3211c46818eb1f3e85d39cb9b 2023-02-13 22:02:06,785 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c8fcdc3211c46818eb1f3e85d39cb9b 2023-02-13 22:02:06,786 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 22:02:06,786 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 22:02:06,868 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:02:06,868 INFO [RS:0;jenkins-hbase20:37979] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,37979,1676325615388; zookeeper connection closed. 2023-02-13 22:02:06,868 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@694c24df rejected from java.util.concurrent.ThreadPoolExecutor@20c3ab8d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:06,868 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:37979-0x1015ddab1e10001, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:02:06,868 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6191a706] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6191a706 2023-02-13 22:02:06,868 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5d73d246 rejected from java.util.concurrent.ThreadPoolExecutor@20c3ab8d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:06,948 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 22:02:06,968 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:02:06,968 INFO [RS:1;jenkins-hbase20:33193] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,33193,1676325615429; zookeeper connection closed. 2023-02-13 22:02:06,968 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@52a8fbe7 rejected from java.util.concurrent.ThreadPoolExecutor@c5c845a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:06,968 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:33193-0x1015ddab1e10002, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:02:06,968 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@23dfc32] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@23dfc32 2023-02-13 22:02:06,968 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@bcfda95 rejected from java.util.concurrent.ThreadPoolExecutor@c5c845a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:07,148 DEBUG [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 22:02:07,197 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/.tmp/table/fabab3fcf26f49c8aea24303ed2fb710 2023-02-13 22:02:07,208 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fabab3fcf26f49c8aea24303ed2fb710 2023-02-13 22:02:07,209 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/.tmp/info/e650a1c18cdb4b54a3833f259e5038b5 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/info/e650a1c18cdb4b54a3833f259e5038b5 2023-02-13 22:02:07,214 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e650a1c18cdb4b54a3833f259e5038b5 2023-02-13 22:02:07,214 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/info/e650a1c18cdb4b54a3833f259e5038b5, entries=59, sequenceid=210, filesize=11.1 K 2023-02-13 22:02:07,215 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/.tmp/rep_barrier/8c8fcdc3211c46818eb1f3e85d39cb9b as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/rep_barrier/8c8fcdc3211c46818eb1f3e85d39cb9b 2023-02-13 22:02:07,219 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c8fcdc3211c46818eb1f3e85d39cb9b 2023-02-13 22:02:07,220 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/rep_barrier/8c8fcdc3211c46818eb1f3e85d39cb9b, entries=24, sequenceid=210, filesize=7.5 K 2023-02-13 22:02:07,221 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/.tmp/table/fabab3fcf26f49c8aea24303ed2fb710 as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/table/fabab3fcf26f49c8aea24303ed2fb710 2023-02-13 22:02:07,225 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fabab3fcf26f49c8aea24303ed2fb710 2023-02-13 22:02:07,225 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/table/fabab3fcf26f49c8aea24303ed2fb710, entries=36, sequenceid=210, filesize=8.0 K 2023-02-13 22:02:07,226 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 477ms, sequenceid=210, compaction requested=false 2023-02-13 22:02:07,232 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-13 22:02:07,233 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 22:02:07,233 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 22:02:07,234 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 22:02:07,234 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676325726624Running coprocessor pre-close hooks at 1676325726624Disabling compacts and flushes for region at 1676325726624Disabling writes for close at 1676325726749 (+125 ms)Obtaining lock to block concurrent updates at 1676325726750 (+1 ms)Preparing flush snapshotting stores in 1588230740 at 1676325726750Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81396, getHeapSize=128792, getOffHeapSize=0, getCellsCount=617 at 1676325726751 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1676325726752 (+1 ms)Flushing 1588230740/info: creating writer at 1676325726752Flushing 1588230740/info: appending metadata at 1676325726755 (+3 ms)Flushing 1588230740/info: closing flushed file at 1676325726755Flushing 1588230740/rep_barrier: creating writer at 1676325726774 (+19 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676325726776 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676325726777 (+1 ms)Flushing 1588230740/table: creating writer at 1676325726785 (+8 ms)Flushing 1588230740/table: appending metadata at 1676325726787 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676325726787Flushing 1588230740/info: reopening flushed file at 1676325727210 (+423 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676325727216 (+6 ms)Flushing 1588230740/table: reopening flushed file at 1676325727221 (+5 ms)Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 477ms, sequenceid=210, compaction requested=false at 1676325727226 (+5 ms)Writing region close event to WAL at 1676325727228 (+2 ms)Running coprocessor post-close hooks at 1676325727233 (+5 ms)Closed at 1676325727234 (+1 ms) 2023-02-13 22:02:07,234 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 22:02:07,348 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,42047,1676325615467; all regions closed. 2023-02-13 22:02:07,358 DEBUG [RS:2;jenkins-hbase20:42047] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/oldWALs 2023-02-13 22:02:07,358 INFO [RS:2;jenkins-hbase20:42047] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C42047%2C1676325615467.meta:.meta(num 1676325617072) 2023-02-13 22:02:07,362 DEBUG [RS:2;jenkins-hbase20:42047] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/oldWALs 2023-02-13 22:02:07,363 INFO [RS:2;jenkins-hbase20:42047] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C42047%2C1676325615467:(num 1676325616937) 2023-02-13 22:02:07,363 DEBUG [RS:2;jenkins-hbase20:42047] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:02:07,363 INFO [RS:2;jenkins-hbase20:42047] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:02:07,363 INFO [RS:2;jenkins-hbase20:42047] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 22:02:07,363 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:02:07,364 INFO [RS:2;jenkins-hbase20:42047] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:42047 2023-02-13 22:02:07,365 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:02:07,365 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,42047,1676325615467 2023-02-13 22:02:07,365 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3b6793e5 rejected from java.util.concurrent.ThreadPoolExecutor@3f581051[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:07,365 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,42047,1676325615467] 2023-02-13 22:02:07,365 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,42047,1676325615467; numProcessing=3 2023-02-13 22:02:07,366 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,42047,1676325615467 already deleted, retry=false 2023-02-13 22:02:07,366 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,42047,1676325615467 expired; onlineServers=0 2023-02-13 22:02:07,366 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,36939,1676325614416' ***** 2023-02-13 22:02:07,366 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 22:02:07,367 DEBUG [M:0;jenkins-hbase20:36939] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70b3e9d5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 22:02:07,367 INFO [M:0;jenkins-hbase20:36939] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,36939,1676325614416 2023-02-13 22:02:07,367 INFO [M:0;jenkins-hbase20:36939] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,36939,1676325614416; all regions closed. 2023-02-13 22:02:07,367 DEBUG [M:0;jenkins-hbase20:36939] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:02:07,367 DEBUG [M:0;jenkins-hbase20:36939] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 22:02:07,367 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 22:02:07,367 DEBUG [M:0;jenkins-hbase20:36939] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 22:02:07,367 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325616572] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325616572,5,FailOnTimeoutGroup] 2023-02-13 22:02:07,367 INFO [M:0;jenkins-hbase20:36939] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 22:02:07,367 INFO [M:0;jenkins-hbase20:36939] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 22:02:07,367 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325616572] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325616572,5,FailOnTimeoutGroup] 2023-02-13 22:02:07,368 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 22:02:07,368 INFO [M:0;jenkins-hbase20:36939] hbase.ChoreService(369): Chore service for: master/jenkins-hbase20:0 had [] on shutdown 2023-02-13 22:02:07,368 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:02:07,368 DEBUG [M:0;jenkins-hbase20:36939] master.HMaster(1502): Stopping service threads 2023-02-13 22:02:07,368 INFO [M:0;jenkins-hbase20:36939] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 22:02:07,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:02:07,368 ERROR [M:0;jenkins-hbase20:36939] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-13 22:02:07,368 INFO [M:0;jenkins-hbase20:36939] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:02:07,368 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 22:02:07,369 DEBUG [M:0;jenkins-hbase20:36939] zookeeper.ZKUtil(398): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 22:02:07,369 WARN [M:0;jenkins-hbase20:36939] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 22:02:07,369 INFO [M:0;jenkins-hbase20:36939] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:02:07,369 INFO [M:0;jenkins-hbase20:36939] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 22:02:07,369 DEBUG [M:0;jenkins-hbase20:36939] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 22:02:07,369 INFO [M:0;jenkins-hbase20:36939] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:02:07,369 DEBUG [M:0;jenkins-hbase20:36939] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:02:07,369 DEBUG [M:0;jenkins-hbase20:36939] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 22:02:07,369 DEBUG [M:0;jenkins-hbase20:36939] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:02:07,369 INFO [M:0;jenkins-hbase20:36939] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.59 KB heapSize=719.56 KB 2023-02-13 22:02:07,466 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:02:07,466 INFO [RS:2;jenkins-hbase20:42047] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,42047,1676325615467; zookeeper connection closed. 2023-02-13 22:02:07,466 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4c959cc7 rejected from java.util.concurrent.ThreadPoolExecutor@3f581051[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:07,466 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): regionserver:42047-0x1015ddab1e10003, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:02:07,466 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@751d1f45] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@751d1f45 2023-02-13 22:02:07,466 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@344f9daf rejected from java.util.concurrent.ThreadPoolExecutor@3f581051[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:07,466 INFO [Listener at localhost.localdomain/42295] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-13 22:02:07,786 INFO [M:0;jenkins-hbase20:36939] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.59 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/6aea97569d3a447fad09baba3a2ce12d 2023-02-13 22:02:07,796 INFO [M:0;jenkins-hbase20:36939] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6aea97569d3a447fad09baba3a2ce12d 2023-02-13 22:02:07,797 DEBUG [M:0;jenkins-hbase20:36939] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/6aea97569d3a447fad09baba3a2ce12d as hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6aea97569d3a447fad09baba3a2ce12d 2023-02-13 22:02:07,802 INFO [M:0;jenkins-hbase20:36939] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6aea97569d3a447fad09baba3a2ce12d 2023-02-13 22:02:07,803 INFO [M:0;jenkins-hbase20:36939] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37463/user/jenkins/test-data/f06b260f-08b2-4551-5683-0fad34bd2a4c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6aea97569d3a447fad09baba3a2ce12d, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-13 22:02:07,804 INFO [M:0;jenkins-hbase20:36939] regionserver.HRegion(2947): Finished flush of dataSize ~608.59 KB/623200, heapSize ~719.55 KB/736816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 434ms, sequenceid=1282, compaction requested=false 2023-02-13 22:02:07,804 INFO [M:0;jenkins-hbase20:36939] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:02:07,804 DEBUG [M:0;jenkins-hbase20:36939] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676325727369Disabling compacts and flushes for region at 1676325727369Disabling writes for close at 1676325727369Obtaining lock to block concurrent updates at 1676325727369Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676325727369Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=623200, getHeapSize=736816, getOffHeapSize=0, getCellsCount=1499 at 1676325727370 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676325727370Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676325727370Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676325727374 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676325727374Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676325727798 (+424 ms)Finished flush of dataSize ~608.59 KB/623200, heapSize ~719.55 KB/736816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 434ms, sequenceid=1282, compaction requested=false at 1676325727804 (+6 ms)Writing region close event to WAL at 1676325727804Closed at 1676325727804 2023-02-13 22:02:07,809 INFO [M:0;jenkins-hbase20:36939] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 22:02:07,809 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:02:07,810 INFO [M:0;jenkins-hbase20:36939] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:36939 2023-02-13 22:02:07,811 DEBUG [M:0;jenkins-hbase20:36939] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase20.apache.org,36939,1676325614416 already deleted, retry=false 2023-02-13 22:02:07,912 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:02:07,912 INFO [M:0;jenkins-hbase20:36939] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,36939,1676325614416; zookeeper connection closed. 2023-02-13 22:02:07,912 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@63d97a76 rejected from java.util.concurrent.ThreadPoolExecutor@6c84c641[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:07,912 DEBUG [Listener at localhost.localdomain/42295-EventThread] zookeeper.ZKWatcher(600): master:36939-0x1015ddab1e10000, quorum=127.0.0.1:63985, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:02:07,912 ERROR [Listener at localhost.localdomain/42295-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@155e6419 rejected from java.util.concurrent.ThreadPoolExecutor@6c84c641[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:02:07,915 WARN [Listener at localhost.localdomain/42295] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 22:02:07,920 INFO [Listener at localhost.localdomain/42295] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 22:02:08,036 WARN [BP-109365581-148.251.75.209-1676325611364 heartbeating to localhost.localdomain/127.0.0.1:37463] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 22:02:08,036 WARN [BP-109365581-148.251.75.209-1676325611364 heartbeating to localhost.localdomain/127.0.0.1:37463] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-109365581-148.251.75.209-1676325611364 (Datanode Uuid afe7de91-8b5d-4950-ba1d-ef3326012043) service to localhost.localdomain/127.0.0.1:37463 2023-02-13 22:02:08,039 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/dfs/data/data5/current/BP-109365581-148.251.75.209-1676325611364] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:02:08,040 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/dfs/data/data6/current/BP-109365581-148.251.75.209-1676325611364] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:02:08,042 WARN [Listener at localhost.localdomain/42295] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 22:02:08,044 INFO [Listener at localhost.localdomain/42295] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 22:02:08,155 WARN [BP-109365581-148.251.75.209-1676325611364 heartbeating to localhost.localdomain/127.0.0.1:37463] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-109365581-148.251.75.209-1676325611364 (Datanode Uuid 643297d9-b5a4-472e-97c3-eb890073d22e) service to localhost.localdomain/127.0.0.1:37463 2023-02-13 22:02:08,157 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/dfs/data/data3/current/BP-109365581-148.251.75.209-1676325611364] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:02:08,157 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/dfs/data/data4/current/BP-109365581-148.251.75.209-1676325611364] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:02:08,158 WARN [Listener at localhost.localdomain/42295] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 22:02:08,161 INFO [Listener at localhost.localdomain/42295] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 22:02:08,263 WARN [BP-109365581-148.251.75.209-1676325611364 heartbeating to localhost.localdomain/127.0.0.1:37463] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-109365581-148.251.75.209-1676325611364 (Datanode Uuid 3f4de7bc-de25-42a6-88c4-90335eaeb4a2) service to localhost.localdomain/127.0.0.1:37463 2023-02-13 22:02:08,264 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/dfs/data/data1/current/BP-109365581-148.251.75.209-1676325611364] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:02:08,264 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7cd72afa-f9cb-5a9a-4e1e-95863ed7cd8b/cluster_cfb51a1e-be56-1a11-cb81-7bf1b19757d3/dfs/data/data2/current/BP-109365581-148.251.75.209-1676325611364] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:02:08,290 INFO [Listener at localhost.localdomain/42295] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-02-13 22:02:08,468 INFO [Listener at localhost.localdomain/42295] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 22:02:08,683 INFO [Listener at localhost.localdomain/42295] hbase.HBaseTestingUtility(1293): Minicluster is down